Benchmark

non-incremental/QF_BV/sage/app7/bench_2022.smt2 

Patrice Godefroid, SAGE (systematic dynamic test generation)
 For more information: http://research.microsoft.com/en-us/um/people/pg/public_psfiles/ndss2008.pdf
Benchmark
Size983793
Compressed Size83161
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2009-08-02
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status unsat
Inferred Status unsat
Size 983785
Compressed Size83169
Max. Term Depth3907
Asserts 1
Declared Functions0
Declared Constants42
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

true1 ite10 not1416 and1
=1927 let3682 extract1157 bvand8837
bvor5657 bvxor1305 bvadd10220 bvmul1
bvsub3311 bvult416 bvule417 bvslt185
bvsle96 bvshl3162 bvashr5380 zero_extend880

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT Evaluation 2013 4Simp 4Simp-SMT-COMP-2012 default unsat ✅ 0.14927 —
Abziz abziz_portfolio_2011_minfeatures_2 default unsat ✅ 0.18905 —
abziz_portfolio_2011_solvers_2 default unsat ✅ 0.18849 —
abziz_portfolio_2012_minfeatures default unsat ✅ 0.18738 —
abziz_portfolio_2012_solvers default unsat ✅ 0.19027 —
AbzizPortfolio-SMT-COMP-2012 default unsat ✅ 0.19035 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default unsat ✅ 0.17782 —
Boolector-SMT-COMP-2011 default unsat ✅ 0.23274 —
Boolector-SMT-COMP-2012 default unsat ✅ 0.18806 —
CVC3 CVC3-SMT-COMP-2010 default unsat ✅ 28.86170 —
CVC3-SMT-COMP-2011 default unsat ✅ 24.44130 —
CVC3-SMT-COMP-2012 default unsat ✅ 25.52400 —
CVC4 CVC4-SMT-COMP-2012-Resubmission default unsat ✅ 0.88979 —
CVC4-SMT-EVAL-2013 default unsat ✅ 1.45919 —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default unsat ✅ 0.30228 —
MathSAT5-HeavyBV-SMT-COMP-2012 default unsat ✅ 0.28358 —
MathSAT5-SMT-COMP-2011 default unsat ✅ 0.37927 —
MathSAT5-SMT-COMP-2012 default unsat ✅ 0.36674 —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default unsat ✅ 0.26810 —
SONOLAR-SMT-COMP-2010 default unsat ✅ 1.52778 —
SONOLAR-SMT-COMP-2011 default unsat ✅ 1.72573 —
SONOLAR-SMT-COMP-2012 default unsat ✅ 0.39660 —
STP STP2-SMT-COMP-2011 default unsat ✅ 0.15006 —
STP2-SMT-COMP-2012 default unsat ✅ 0.14107 —
simplifyingSTP-SMT-COMP-2010 default unsat ✅ 0.13909 —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default unsat ✅ 0.19080 —
Z3-SMT-COMP-2011 default unsat ✅ 0.19297 —
SMT-COMP 2015 0.14 (6/7) Boolector Boolector SMT15 QF_BV final boolector_qf_bv unsat ✅ 0.24769 0.24696
CVC4 CVC4-master-2015-06-15-9b32405-main default unsat ✅ 1.38482 2.27765
CVC4-experimental-2015-06-15-ff5745a-main default unsat ✅ 1.35550 2.24066
MathSAT MathSat 5.3.6 main smtcomp2015_main unsat ✅ 0.21838 0.21697
SMT-RAT SMT-RAT-final default unknown ❌ 2400.01000 2400.80000
STP stp-cryptominisat4 default unsat ✅ 0.12986 0.12898
stp-cmsat4-v15 default unsat ✅ 0.12911 0.12798
stp-cmsat4-mt-v15 default unsat ✅ 0.12871 0.12798
stp-minisat-v15 default unsat ✅ 0.12809 0.12698
Yices2 Yices default unsat ✅ 0.06201 0.06099
Z3 z3 4.4.0 default unsat ✅ 0.18836 0.18797
SMT-COMP 2016 ABC ABC_default default_abc unsat ✅ 0.56541 0.56565
ABC_glucose glucose unsat ✅ 0.56011 0.56001
Boolector Boolector boolector unsat ✅ 0.27811 0.27821
Boolector preprop boolector unsat ✅ 0.27231 0.27283
CVC4 CVC4-master-2016-05-27-cfef263-main default unsat ✅ 1.31167 2.16238
MapleSTP MapleSTP default unsat ✅ 0.12144 0.12161
MapleSTP-mt default unsat ✅ 0.12162 0.12162
MathSAT mathsat-5.3.11-linux-x86_64-Main default unsat ✅ 0.19571 0.19709
MinkeyRink Minkeyrink 2016 default unsat ✅ 0.16888 0.16895
Q3B Q3B default unsat ✅ 0.36887 0.83818
STP stp-cms-exp-2016 default unsat ✅ 0.12537 0.12537
stp-cms-mt-2016 default unsat ✅ 0.12806 0.12811
stp-cms-st-2016 default unsat ✅ 0.12666 0.12672
stp-minisat-st-2016 default unsat ✅ 0.12757 0.12760
Yices2 Yices-2.4.2 default unsat ✅ 0.06262 0.06282
Z3 z3-4.4.1 default unsat ✅ 0.21341 0.21481
SMT-COMP 2017 Boolector Boolector+CaDiCaL SMT17 final boolector unsat ✅ 0.23321 0.22879
Boolector SMT17 final boolector unsat ✅ 0.25584 0.25546
CVC4 CVC4-smtcomp2017-main default unsat ✅ 1.40339 2.30280
MathSAT mathsat-5.4.1-linux-x86_64-Main default unsat ✅ 0.20151 0.20127
MinkeyRink MinkeyRink 2017.3a default unsat ✅ 0.40222 0.40121
Q3B Q3B default unsat ✅ 0.35777 0.79911
STP stp_st default unsat ✅ 0.15778 0.15697
stp_mt default unsat ✅ 0.16247 0.16029
Yices2 Yices2-Main default unsat ✅ 0.06227 0.06077
Z3 z3-4.5.0 default unsat ✅ 0.20089 0.20002
SMT-COMP 2018 Boolector Boolector_default unsat ✅ 0.20700 0.20671
CVC4 master-2018-06-10-b19c840-competition-default_default unsat ✅ 1.21380 1.21369
MathSAT mathsat-5.5.2-linux-x86_64-Main_default unsat ✅ 0.19195 0.19311
MinkeyRink Minkeyrink MT_mt unsat ✅ 0.38207 0.38196
Minkeyrink ST_st unsat ✅ 0.38215 0.38212
STP STP-CMS-st-2018_default-no-stderr unsat ✅ 0.15719 0.15678
STP-CMS-mt-2018_multicore-no-stderr unsat ✅ 0.16557 0.16561
STP-Riss-st-2018_riss-no-stderr unsat ✅ 0.15700 0.15702
Yices2 Yices 2.6.0_default unsat ✅ 0.05999 0.05989
Z3 z3-4.7.1_default unsat ✅ 0.22715 0.22702
SMT-COMP 2019 Boolector Boolector-wrapped-sq_default unsat ✅ 0.37437 0.37436
Poolector-wrapped-sq_default unsat ✅ 0.58324 1.76419
Boolector-ReasonLS boolector-ReasonLS-wrapped-sq_default unsat ✅ 0.30599 0.30595
CVC4 CVC4-2019-06-03-d350fe1-wrapped-sq_default unsat ✅ 1.20994 1.21007
MinkeyRink MinkeyRink MT-wrapped-sq_default unsat ✅ 0.30954 0.30970
MinkeyRink ST-wrapped-sq_default unsat ✅ 0.30560 0.30565
Par4 Par4-wrapped-sq_default unsat ✅ 0.06743 0.00614
STP STP-2019-wrapped-sq_default unsat ✅ 0.17019 0.17035
stp-mergesat-fixed-wrapped-sq_default unsat ✅ 0.16987 0.17004
stp-minisat-wrapped-sq_default unsat ✅ 0.17106 0.17123
stp-mt-wrapped-sq_default unsat ✅ 0.16984 0.16992
stp-portfolio-fixed-wrapped-sq_default unsat ✅ 0.16881 0.16893
stp-riss-wrapped-sq_default unsat ✅ 0.17028 0.17042
Yices2 Yices 2.6.2-wrapped-sq_default unsat ✅ 0.06260 0.06259
Yices 2.6.2 Cadical-wrapped-sq_default unsat ✅ 0.06129 0.06127
Yices 2.6.2 Cryptominisat-wrapped-sq_default unsat ✅ 0.06325 0.06330
Yices 2.6.2 MCSAT BV-wrapped-sq_default unsat ✅ 0.06163 0.06159
Yices 2.6.2 new bvsolver-wrapped-sq_default unsat ✅ 0.06164 0.06166
Z3 z3-4.8.4-d6df51951f4c-wrapped-sq_default unsat ✅ 0.23461 0.23459
SMT-COMP 2021 Bitwuzla Bitwuzla-fixed_default unsat ✅ 0.31099 0.31088
MathSAT mathsat-5.6.6_default unsat ✅ 0.20432 0.20422
STP STP 2021.0_default unsat ✅ 0.16194 0.16204
Z3 z3-4.8.11_default unsat ✅ 0.18020 0.18013
SMT-COMP 2022 Bitwuzla Bitwuzla-fixed_default unsat ✅ 0.31305 0.31301
Bitwuzla-wrapped_default unsat ✅ 0.34872 0.34857
cvc5 cvc5-default-2022-07-02-b15e116-wrapped_sq unsat ✅ 1.23262 1.23305
MathSAT MathSAT-5.6.8_default unsat ✅ 0.20361 0.20353
STP STP 2022.4_default unsat ✅ 0.17123 0.17121
Yices2 Yices 2.6.2 for SMTCOMP 2021_default unsat ✅ 0.07134 0.07127
Z3++BV z3++bv_0702_default unsat ✅ 0.18534 0.18527
Z3 z3-4.8.17_default unsat ✅ 0.17537 0.17720
SMT-COMP 2024 Bitwuzla Bitwuzla unsat ✅ 0.49945 0.39951
cvc5 cvc5 unsat ✅ 0.55951 0.46050
SMTInterpol SMTInterpol unsat ✅ 708.62048 1050.13660
STP STP unsat ✅ 3.37762 3.27641
Yices2 Yices2 unsat ✅ 0.25924 0.15941
Z3alpha Z3-alpha unsat ✅ 0.40463 0.30444
SMT-COMP 2025 0.11 (8/9) Bitwuzla Bitwuzla unsat ✅ 0.48342 0.35096
Bitwuzla-MachBV-base unsat ✅ 0.49100 0.35780
Bitwuzla-MachBV Bitwuzla-MachBV unsat ✅ 0.58620 0.46642
BVDecide bv_decide unsat ✅ 140.05615 139.92061
bv_decide-nokernel unsat ✅ 119.98394 119.82420
cvc5 cvc5 unsat ✅ 0.47600 0.35759
SMTInterpol SMTInterpol unknown ❌ 1201.89681 3617.56256
Yices2 Yices2 unsat ✅ 0.31592 0.19348
Z3alpha Z3-alpha unsat ✅ 0.50659 0.40815
Z3 Z3-alpha-base unsat ✅ 0.37939 0.25677
Z3-Owl-base unsat ✅ 0.73246 0.60357
z3siri-base unsat ✅ 0.37831 0.25224
Z3-Owl Z3-Owl unsat ✅ 0.92543 0.79312