Benchmark

non-incremental/QF_BV/sage/app5/bench_1016.smt2 

Patrice Godefroid, SAGE (systematic dynamic test generation)
 For more information: http://research.microsoft.com/en-us/um/people/pg/public_psfiles/ndss2008.pdf
Benchmark
Size12896
Compressed Size2097
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2009-08-02
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status unsat
Inferred Status unsat
Size 12888
Compressed Size2081
Max. Term Depth48
Asserts 1
Declared Functions0
Declared Constants48
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

true1 ite22 not36 and1
=50 let38 bvor28 bvadd55
bvsub52 bvult34 bvule88 bvsle12
bvshl28 zero_extend61

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT Evaluation 2013 4Simp 4Simp-SMT-COMP-2012 default unsat ✅ 0.10675 —
Abziz abziz_portfolio_2011_minfeatures_2 default unsat ✅ 0.13190 —
abziz_portfolio_2011_solvers_2 default unsat ✅ 0.12945 —
abziz_portfolio_2012_minfeatures default unsat ✅ 0.13187 —
abziz_portfolio_2012_solvers default unsat ✅ 0.13053 —
AbzizPortfolio-SMT-COMP-2012 default unsat ✅ 0.13144 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default unsat ✅ 0.02984 —
Boolector-SMT-COMP-2011 default unsat ✅ 0.20518 —
Boolector-SMT-COMP-2012 default unsat ✅ 0.03971 —
CVC3 CVC3-SMT-COMP-2010 default unsat ✅ 1.35317 —
CVC3-SMT-COMP-2011 default unsat ✅ 1.26711 —
CVC3-SMT-COMP-2012 default unsat ✅ 1.27766 —
CVC4 CVC4-SMT-COMP-2012-Resubmission default unsat ✅ 0.05062 —
CVC4-SMT-EVAL-2013 default unsat ✅ 0.06156 —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default unsat ✅ 0.07444 —
MathSAT5-HeavyBV-SMT-COMP-2012 default unsat ✅ 0.59425 —
MathSAT5-SMT-COMP-2011 default unsat ✅ 0.09367 —
MathSAT5-SMT-COMP-2012 default unsat ✅ 0.06567 —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default unsat ✅ 0.06741 —
SONOLAR-SMT-COMP-2010 default unsat ✅ 0.60545 —
SONOLAR-SMT-COMP-2011 default unsat ✅ 0.07771 —
SONOLAR-SMT-COMP-2012 default unsat ✅ 0.06081 —
STP STP2-SMT-COMP-2011 default unsat ✅ 0.03494 —
STP2-SMT-COMP-2012 default unsat ✅ 0.08018 —
simplifyingSTP-SMT-COMP-2010 default unsat ✅ 0.05848 —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default unsat ✅ 0.12371 —
Z3-SMT-COMP-2011 default unsat ✅ 0.12510 —
SMT-COMP 2015 Boolector Boolector SMT15 QF_BV final boolector_qf_bv unsat ✅ 0.09452 0.09398
CVC4 CVC4-master-2015-06-15-9b32405-main default unsat ✅ 0.09011 0.15898
CVC4-experimental-2015-06-15-ff5745a-main default unsat ✅ 0.08747 0.15498
MathSAT MathSat 5.3.6 main smtcomp2015_main unsat ✅ 0.20984 0.20897
SMT-RAT SMT-RAT-final default unsat ✅ 11.23150 11.23530
STP stp-cryptominisat4 default unsat ✅ 0.05434 0.05299
stp-cmsat4-v15 default unsat ✅ 0.05021 0.04899
stp-cmsat4-mt-v15 default unknown ❌ 0.05697 0.05599
stp-minisat-v15 default unsat ✅ 0.06033 0.05899
Yices2 Yices default unsat ✅ 0.51246 0.51192
Z3 z3 4.4.0 default unsat ✅ 0.10034 0.09998
SMT-COMP 2016 ABC ABC_default default_abc unsat ✅ 0.16947 0.16951
ABC_glucose glucose unsat ✅ 1.08392 1.08465
Boolector Boolector boolector unsat ✅ 0.06890 0.06890
Boolector preprop boolector unsat ✅ 0.07497 0.07498
CVC4 CVC4-master-2016-05-27-cfef263-main default unsat ✅ 0.08304 0.14651
MapleSTP MapleSTP default unsat ✅ 0.05537 0.05534
MapleSTP-mt default unsat ✅ 0.06363 0.14464
MathSAT mathsat-5.3.11-linux-x86_64-Main default unsat ✅ 0.19521 0.19623
MinkeyRink Minkeyrink 2016 default unsat ✅ 0.25689 0.35722
Q3B Q3B default unsat ✅ 18.34550 54.62000
STP stp-cms-exp-2016 default unsat ✅ 0.05767 0.05767
stp-cms-mt-2016 default unsat ✅ 0.06131 0.13991
stp-cms-st-2016 default unsat ✅ 0.05434 0.05449
stp-minisat-st-2016 default unsat ✅ 0.06462 0.06462
Yices2 Yices-2.4.2 default unsat ✅ 0.01277 0.00681
Z3 z3-4.4.1 default unsat ✅ 0.09995 0.10118
SMT-COMP 2017 Boolector Boolector+CaDiCaL SMT17 final boolector unsat ✅ 0.10525 0.10434
Boolector SMT17 final boolector unsat ✅ 0.13662 0.13589
CVC4 CVC4-smtcomp2017-main default unsat ✅ 0.08468 0.15034
MathSAT mathsat-5.4.1-linux-x86_64-Main default unsat ✅ 0.18290 0.18228
MinkeyRink MinkeyRink 2017.3a default unsat ✅ 0.11371 0.10119
Q3B Q3B default unsat ✅ 47.40680 141.86000
STP stp_st default unsat ✅ 0.06372 0.06275
stp_mt default unsat ✅ 0.06857 0.14491
Yices2 Yices2-Main default unsat ✅ 0.00839 0.00708
Z3 z3-4.5.0 default unsat ✅ 0.17533 0.17452
SMT-COMP 2018 Boolector Boolector_default unsat ✅ 1.02239 1.02226
CVC4 master-2018-06-10-b19c840-competition-default_default unsat ✅ 0.12995 0.13005
MathSAT mathsat-5.5.2-linux-x86_64-Main_default unsat ✅ 0.18166 0.18261
MinkeyRink Minkeyrink MT_mt unsat ✅ 0.06957 0.09053
Minkeyrink ST_st unsat ✅ 0.06638 0.06662
STP STP-CMS-st-2018_default-no-stderr unsat ✅ 0.04273 0.04293
STP-CMS-mt-2018_multicore-no-stderr unsat ✅ 0.04658 0.06636
STP-Riss-st-2018_riss-no-stderr unsat ✅ 0.19782 0.19784
Yices2 Yices 2.6.0_default unsat ✅ 0.01100 0.00887
Z3 z3-4.7.1_default unsat ✅ 0.15232 0.15227
SMT-COMP 2019 Boolector Boolector-wrapped-sq_default unsat ✅ 0.11960 0.11961
Poolector-wrapped-sq_default unsat ✅ 0.17844 0.21791
Boolector-ReasonLS boolector-ReasonLS-wrapped-sq_default unsat ✅ 0.06910 0.06911
CVC4 CVC4-2019-06-03-d350fe1-wrapped-sq_default unsat ✅ 0.07498 0.07517
MinkeyRink MinkeyRink MT-wrapped-sq_default unsat ✅ 0.06317 0.08150
MinkeyRink ST-wrapped-sq_default unsat ✅ 0.05552 0.05563
Par4 Par4-wrapped-sq_default unsat ✅ 0.02311 0.00606
STP STP-2019-wrapped-sq_default unsat ✅ 0.04978 0.05015
stp-mergesat-fixed-wrapped-sq_default unsat ✅ 0.05826 0.05850
stp-minisat-wrapped-sq_default unsat ✅ 0.04819 0.04834
stp-mt-wrapped-sq_default unsat ✅ 0.05503 0.07049
stp-portfolio-fixed-wrapped-sq_default unsat ✅ 0.08984 0.12502
stp-riss-wrapped-sq_default unknown ❌ 0.04204 0.04225
Yices2 Yices 2.6.2-wrapped-sq_default unsat ✅ 0.00915 0.00916
Yices 2.6.2 Cadical-wrapped-sq_default unsat ✅ 0.01562 0.01253
Yices 2.6.2 Cryptominisat-wrapped-sq_default unsat ✅ 0.01235 0.01239
Yices 2.6.2 MCSAT BV-wrapped-sq_default unsat ✅ 0.01622 0.01624
Yices 2.6.2 new bvsolver-wrapped-sq_default unsat ✅ 0.01196 0.01195
Z3 z3-4.8.4-d6df51951f4c-wrapped-sq_default unsat ✅ 0.13240 0.13238
SMT-COMP 2022 Bitwuzla Bitwuzla-fixed_default unsat ✅ 0.15154 0.15153
Bitwuzla-wrapped_default unsat ✅ 0.15663 0.15657
cvc5 cvc5-default-2022-07-02-b15e116-wrapped_sq unsat ✅ 0.07809 0.07865
MathSAT MathSAT-5.6.8_default unsat ✅ 0.10133 0.10122
STP STP 2022.4_default unsat ✅ 0.05605 0.05603
Yices2 Yices 2.6.2 for SMTCOMP 2021_default unsat ✅ 0.01960 0.00777
Z3++BV z3++bv_0702_default unsat ✅ 0.04580 0.04570
Z3 z3-4.8.17_default unsat ✅ 0.08035 0.08209
SMT-COMP 2024 Bitwuzla Bitwuzla unsat ✅ 0.27012 0.17009
cvc5 cvc5 unsat ✅ 0.25883 0.15928
SMTInterpol SMTInterpol unsat ✅ 2.98479 8.87246
STP STP unsat ✅ 0.26065 0.16076
Yices2 Yices2 unsat ✅ 0.22631 0.12669
Z3alpha Z3-alpha unsat ✅ 0.31590 0.21617
SMT-COMP 2025 Bitwuzla Bitwuzla unsat ✅ 0.27723 0.16185
Bitwuzla-MachBV-base unsat ✅ 0.28385 0.16458
Bitwuzla-MachBV Bitwuzla-MachBV unsat ✅ 0.38860 0.26134
BVDecide bv_decide unsat ✅ 1.89086 1.74808
bv_decide-nokernel unsat ✅ 1.43081 1.26825
cvc5 cvc5 unsat ✅ 0.30392 0.18697
SMTInterpol SMTInterpol unsat ✅ 2.30378 6.70988
Yices2 Yices2 unsat ✅ 0.28670 0.16388
Z3alpha Z3-alpha unsat ✅ 0.41816 0.36203
Z3 Z3-alpha-base unsat ✅ 0.33559 0.20839
Z3-Owl-base unsat ✅ 0.39594 0.27128
z3siri-base unsat ✅ 0.30465 0.18615
Z3-Owl Z3-Owl unsat ✅ 1.03988 0.91670