Benchmark

non-incremental/QF_BV/sage/app1/bench_2004.smt2 

Patrice Godefroid, SAGE (systematic dynamic test generation)
 For more information: http://research.microsoft.com/en-us/um/people/pg/public_psfiles/ndss2008.pdf
Benchmark
Size77031
Compressed Size11047
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2009-08-02
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status sat
Inferred Status sat
Size 77023
Compressed Size11047
Max. Term Depth188
Asserts 1
Declared Functions0
Declared Constants290
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

true1 ite82 not75 and1
=198 let178 extract1 bvand44
bvor174 bvadd378 bvmul5 bvsub253
bvult267 bvule580 bvslt1 bvsle57
bvshl175 bvashr5 zero_extend314

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT Evaluation 2013 0.11 (8/9) 4Simp 4Simp-SMT-COMP-2012 default sat ✅ 1.46262 —
Abziz abziz_portfolio_2011_minfeatures_2 default sat ✅ 5.03340 —
abziz_portfolio_2011_solvers_2 default sat ✅ 7.73724 —
abziz_portfolio_2012_minfeatures default sat ✅ 7.67513 —
abziz_portfolio_2012_solvers default sat ✅ 6.04815 —
AbzizPortfolio-SMT-COMP-2012 default sat ✅ 7.73685 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default sat ✅ 7.00136 —
Boolector-SMT-COMP-2011 default sat ✅ 3.22042 —
Boolector-SMT-COMP-2012 default sat ✅ 0.95649 —
CVC3 CVC3-SMT-COMP-2010 default unknown ❌ — —
CVC3-SMT-COMP-2011 default unknown ❌ — —
CVC3-SMT-COMP-2012 default unknown ❌ — —
CVC4 CVC4-SMT-COMP-2012-Resubmission default sat ✅ 1.92199 —
CVC4-SMT-EVAL-2013 default sat ✅ 3.24712 —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default sat ✅ 2.73367 —
MathSAT5-HeavyBV-SMT-COMP-2012 default sat ✅ 2.92020 —
MathSAT5-SMT-COMP-2011 default sat ✅ 2.65402 —
MathSAT5-SMT-COMP-2012 default sat ✅ 2.62520 —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default sat ✅ 1.15656 —
SONOLAR-SMT-COMP-2010 default sat ✅ 2.57639 —
SONOLAR-SMT-COMP-2011 default sat ✅ 1.66345 —
SONOLAR-SMT-COMP-2012 default sat ✅ 0.94747 —
STP STP2-SMT-COMP-2011 default sat ✅ 4.85982 —
STP2-SMT-COMP-2012 default sat ✅ 9.86472 —
simplifyingSTP-SMT-COMP-2010 default sat ✅ 13.54770 —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default sat ✅ 4.08857 —
Z3-SMT-COMP-2011 default sat ✅ 5.03469 —
SMT-COMP 2015 Boolector Boolector SMT15 QF_BV final boolector_qf_bv sat ✅ 4.42201 4.42233
CVC4 CVC4-master-2015-06-15-9b32405-main default sat ✅ 9.12866 18.25120
CVC4-experimental-2015-06-15-ff5745a-main default sat ✅ 9.02568 18.04030
MathSAT MathSat 5.3.6 main smtcomp2015_main sat ✅ 3.55005 3.55046
SMT-RAT SMT-RAT-final default sat ✅ 476.32600 476.51500
STP stp-cryptominisat4 default sat ✅ 16.62710 16.62950
stp-cmsat4-v15 default sat ✅ 27.30140 27.30580
stp-cmsat4-mt-v15 default unknown ❌ 0.42660 0.42593
stp-minisat-v15 default sat ✅ 4.54450 4.54431
Yices2 Yices default sat ✅ 2.53225 2.53261
Z3 z3 4.4.0 default sat ✅ 5.46150 5.46317
SMT-COMP 2016 0.10 (9/10) ABC ABC_default default_abc sat ✅ 67.43900 67.47720
ABC_glucose glucose sat ✅ 13.10510 25.63230
Boolector Boolector boolector sat ✅ 3.32377 3.32562
Boolector preprop boolector sat ✅ 3.32725 3.32908
CVC4 CVC4-master-2016-05-27-cfef263-main default sat ✅ 10.79540 21.59890
MapleSTP MapleSTP default sat ✅ 515.94200 516.27400
MapleSTP-mt default sat ✅ 5.32113 19.92550
MathSAT mathsat-5.3.11-linux-x86_64-Main default sat ✅ 2.75026 2.75281
MinkeyRink Minkeyrink 2016 default sat ✅ 3.91123 11.17330
Q3B Q3B default unknown ❌ 2400.03000 7204.10000
STP stp-cms-exp-2016 default sat ✅ 64.98220 65.02490
stp-cms-mt-2016 default sat ✅ 12.41280 48.15110
stp-cms-st-2016 default sat ✅ 43.04610 43.07050
stp-minisat-st-2016 default sat ✅ 26.70770 26.72300
Yices2 Yices-2.4.2 default sat ✅ 1.73726 1.73823
Z3 z3-4.4.1 default sat ✅ 6.41388 6.41964
SMT-COMP 2017 0.12 (7/8) Boolector Boolector+CaDiCaL SMT17 final boolector sat ✅ 1.47504 1.47442
Boolector SMT17 final boolector sat ✅ 3.63743 3.63143
CVC4 CVC4-smtcomp2017-main default sat ✅ 7.91422 15.81270
MathSAT mathsat-5.4.1-linux-x86_64-Main default sat ✅ 3.05751 3.05602
MinkeyRink MinkeyRink 2017.3a default sat ✅ 1.83451 1.85768
Q3B Q3B default unknown ❌ 600.05200 1787.71000
STP stp_st default sat ✅ 10.74470 10.74300
stp_mt default sat ✅ 4.78629 17.41590
Yices2 Yices2-Main default sat ✅ 0.89305 0.89160
Z3 z3-4.5.0 default sat ✅ 8.35348 8.35049
SMT-COMP 2018 Boolector Boolector_default sat ✅ 0.94425 0.94413
CVC4 master-2018-06-10-b19c840-competition-default_default sat ✅ 6.79681 6.79716
MathSAT mathsat-5.5.2-linux-x86_64-Main_default sat ✅ 3.14253 3.14303
MinkeyRink Minkeyrink MT_mt sat ✅ 2.63221 9.60911
Minkeyrink ST_st sat ✅ 4.47467 4.47462
STP STP-CMS-st-2018_default-no-stderr sat ✅ 12.03850 12.03730
STP-CMS-mt-2018_multicore-no-stderr sat ✅ 15.05300 58.37640
STP-Riss-st-2018_riss-no-stderr sat ✅ 11.95800 11.95730
Yices2 Yices 2.6.0_default sat ✅ 2.16109 2.16082
Z3 z3-4.7.1_default sat ✅ 8.62221 8.62161
SMT-COMP 2019 Boolector Boolector-wrapped-sq_default sat ✅ 1.33615 1.33614
Poolector-wrapped-sq_default sat ✅ 1.59784 5.96000
Boolector-ReasonLS boolector-ReasonLS-wrapped-sq_default sat ✅ 3.84294 7.61183
CVC4 CVC4-2019-06-03-d350fe1-wrapped-sq_default sat ✅ 13.88820 13.88840
MinkeyRink MinkeyRink MT-wrapped-sq_default sat ✅ 1.95780 6.94807
MinkeyRink ST-wrapped-sq_default sat ✅ 2.06635 2.06629
Par4 Par4-wrapped-sq_default sat ✅ 0.77641 2.21000
STP STP-2019-wrapped-sq_default sat ✅ 13.77870 13.77910
stp-mergesat-fixed-wrapped-sq_default sat ✅ 11.40880 11.40890
stp-minisat-wrapped-sq_default sat ✅ 5.85492 5.85410
stp-mt-wrapped-sq_default sat ✅ 16.01550 62.11690
stp-portfolio-fixed-wrapped-sq_default sat ✅ 14.76920 32.83610
stp-riss-wrapped-sq_default unknown ❌ 0.51534 0.51535
Yices2 Yices 2.6.2-wrapped-sq_default sat ✅ 3.23710 3.23686
Yices 2.6.2 Cadical-wrapped-sq_default sat ✅ 1.26324 1.26312
Yices 2.6.2 Cryptominisat-wrapped-sq_default sat ✅ 4.57966 4.57927
Yices 2.6.2 MCSAT BV-wrapped-sq_default sat ✅ 32.86550 32.86590
Yices 2.6.2 new bvsolver-wrapped-sq_default sat ✅ 1.44265 1.44254
Z3 z3-4.8.4-d6df51951f4c-wrapped-sq_default sat ✅ 7.61561 7.61520
SMT-COMP 2020 0.11 (8/9) Bitwuzla Bitwuzla-fixed_default sat ✅ 1.39773 1.39736
Boolector Boolector-wrapped-sq_default sat ✅ 0.31809 0.31810
Poolector-wrapped-sq_default sat ✅ 0.48268 0.44635
CVC4 CVC4-sq-final_default sat ✅ 7.77701 7.77611
LazyBV2Int LazyBV2Int20200523_default.sh unknown ❌ 1200.08000 1199.88000
MathSAT MathSAT5_default.sh sat ✅ 1.34338 1.34330
MinkeyRink MinkeyRink Solver 2020.3.1_default unknown ❌ 0.30602 0.28890
MinkeyRink Solver 2020.3_default sat ✅ 0.87365 2.60319
STP STP_default sat ✅ 4.10874 14.91570
STP ++ Mergsat v1_default sat ✅ 17.53220 17.52720
Yices2 Yices 2.6.2 bug fix_default sat ✅ 0.97891 0.97869
Z3 z3-4.8.8_default sat ✅ 3.66396 3.66365
SMT-COMP 2022 Bitwuzla Bitwuzla-fixed_default sat ✅ 1.29743 1.29724
Bitwuzla-wrapped_default sat ✅ 0.55043 0.55022
cvc5 cvc5-default-2022-07-02-b15e116-wrapped_sq sat ✅ 7.49435 7.49274
MathSAT MathSAT-5.6.8_default sat ✅ 2.35825 2.35812
STP STP 2022.4_default sat ✅ 1.04566 1.04551
Yices2 Yices 2.6.2 for SMTCOMP 2021_default sat ✅ 0.97751 1.88742
Z3++BV z3++bv_0702_default sat ✅ 4.24151 4.24089
Z3 z3-4.8.17_default sat ✅ 5.04115 5.03989
SMT-COMP 2024 0.17 (5/6) Bitwuzla Bitwuzla sat ✅ 1.26366 1.16378
cvc5 cvc5 sat ✅ 7.39115 7.29132
SMTInterpol SMTInterpol unknown ❌ 1201.71940 1235.92721
STP STP sat ✅ 0.97911 0.86662
Yices2 Yices2 sat ✅ 0.94578 0.84714
Z3alpha Z3-alpha sat ✅ 2.38935 2.28931
SMT-COMP 2025 0.11 (8/9) Bitwuzla Bitwuzla sat ✅ 0.54126 0.41281
Bitwuzla-MachBV-base sat ✅ 1.18246 1.06076
Bitwuzla-MachBV Bitwuzla-MachBV sat ✅ 0.61167 0.48594
BVDecide bv_decide sat ✅ 9.91099 9.74786
bv_decide-nokernel sat ✅ 9.36631 9.21045
cvc5 cvc5 sat ✅ 10.83093 10.70515
SMTInterpol SMTInterpol unknown ❌ 1201.58078 1227.61937
Yices2 Yices2 sat ✅ 1.10925 0.99356
Z3alpha Z3-alpha sat ✅ 16.44528 62.95273
Z3 Z3-alpha-base sat ✅ 16.00707 15.88617
Z3-Owl-base sat ✅ 73.30934 73.16393
z3siri-base sat ✅ 16.19995 16.08004
Z3-Owl Z3-Owl sat ✅ 9.91440 9.77838