Benchmark

non-incremental/QF_BV/tacas07/Y86_btnft.smt2 

This directory contains benchmarks used in the experimental results
reported in the following paper:

Title: Deciding Bit-Vector Arithmetic with Abstraction
Authors: R. E. Bryant, D. Kroening, J. Ouaknine, S. A. Seshia, O. Strichman, B. Brady

If you use these benchmarks, we will appreciate hearing from you by 
e-mail to sseshia@eecs.berkeley.edu. Additionally, the following sources 
must be acknowledged:

1) Y86_std.smt, Y86_btnft.smt: 
   Generated from a UCLID model created by R. E. Bryant, as described
   in the following paper:
    http://www.cs.cmu.edu/~bryant/pubdir/CMU-CS-05-195.pdf

2) s-40-50-bv.smt: 
   Generated by V. Ganapathy for finding format-string exploits in
   the WiSA project (http://www.cs.wisc.edu/wisa)
 
3) BBB-32.smt, rfunit_flat-64.smt:
   Generated from benchmarks generated by Z. Hanna of Intel.

All benchmarks are in the SMT-LIB format, as described at:
http://combination.cs.uiowa.edu/smtlib/
Benchmark
Size186660
Compressed Size29468
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2007-07-03
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status unsat
Inferred Status unsat
Size 186652
Compressed Size29109
Max. Term Depth2033
Asserts 1
Declared Functions0
Declared Constants247
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

ite758 not1710 or2370 and2617
=3980 let348 extract12 bvneg1
bvadd78 bvult37

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT-COMP 2010 0.67 (1/3) CVC3 CVC3 2.3 unknown ❌ 1200.00000 —
SONOLAR SONOLAR r252 unknown ❌ 1200.00000 —
STP simplifyingSTP unsat ✅ 815.50000 —
SMT-COMP 2012 0.43 (4/7) 4Simp 4Simp unsat ✅ 104.20000 —
Abziz AbzizPortfolio+BOOLECTOR+MATHSAT+SONOLAR+STP2+Z3_6 unknown ❌ 1200.00000 —
Boolector Boolector unsat ✅ 337.30000 —
CVC3 CVC3 v2.4.2 unknown ❌ — —
CVC4 1.0rc.3931 unsat ✅ 79.40000 —
MathSAT MathSAT-HeavyBV unknown ❌ 1200.20000 —
SONOLAR SONOLAR unsat ✅ 171.90000 —
Z3 Z3 unknown ❌ 1200.00000 —
SMT Evaluation 2013 0.78 (2/9) 4Simp 4Simp-SMT-COMP-2012 default unknown ❌ — —
Abziz abziz_portfolio_2011_minfeatures_2 default unsat ✅ 108.84800 —
abziz_portfolio_2011_solvers_2 default unsat ✅ 22.18320 —
abziz_portfolio_2012_minfeatures default unsat ✅ — —
abziz_portfolio_2012_solvers default unknown ❌ — —
AbzizPortfolio-SMT-COMP-2012 default unsat ✅ 78.27780 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default unknown ❌ — —
Boolector-SMT-COMP-2011 default unknown ❌ — —
Boolector-SMT-COMP-2012 default unknown ❌ — —
CVC3 CVC3-SMT-COMP-2010 default unknown ❌ — —
CVC3-SMT-COMP-2011 default unknown ❌ — —
CVC3-SMT-COMP-2012 default unknown ❌ — —
CVC4 CVC4-SMT-COMP-2012-Resubmission default unknown ❌ — —
CVC4-SMT-EVAL-2013 default unknown ❌ — —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default unknown ❌ — —
MathSAT5-HeavyBV-SMT-COMP-2012 default unknown ❌ — —
MathSAT5-SMT-COMP-2011 default unknown ❌ — —
MathSAT5-SMT-COMP-2012 default unknown ❌ — —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default unknown ❌ — —
SONOLAR-SMT-COMP-2010 default unsat ✅ 481.47500 —
SONOLAR-SMT-COMP-2011 default unknown ❌ — —
SONOLAR-SMT-COMP-2012 default unknown ❌ — —
STP STP2-SMT-COMP-2011 default unknown ❌ — —
STP2-SMT-COMP-2012 default unknown ❌ — —
simplifyingSTP-SMT-COMP-2010 default unknown ❌ — —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default unknown ❌ — —
Z3-SMT-COMP-2011 default unknown ❌ — —
SMT-COMP 2014 4Simp 4Simp - 2014 default unsat ✅ 50.71360 50.72630
Abziz abziz_portfolio_all_features default unsat ✅ 107.62700 107.52000
abziz_portfolio_min_features default unsat ✅ 104.53400 104.46500
Boolector Boolector boolector unsat ✅ 120.94300 120.98700
CVC4 CVC4 f7118b2 default unsat ✅ 155.35000 311.63500
MathSAT MathSAT-5.2.12-Main default unsat ✅ 15.63510 15.63160
SONOLAR sonolar_smtcomp-2014 default unsat ✅ 63.19060 63.20940
STP stp-cryptominisat4 default unsat ✅ 289.44700 289.51200
Yices2 Yices-2.2.1-smtcomp2014 default unsat ✅ 1445.03000 1445.73000
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-COMP-2014 default unsat ✅ 462.52600 462.58600
SMT-COMP 2015 0.14 (6/7) Boolector Boolector SMT15 QF_BV final boolector_qf_bv unsat ✅ 78.60920 78.62600
CVC4 CVC4-master-2015-06-15-9b32405-main default unsat ✅ 108.38600 217.54800
CVC4-experimental-2015-06-15-ff5745a-main default unsat ✅ 109.30300 219.38100
MathSAT MathSat 5.3.6 main smtcomp2015_main unsat ✅ 13.73030 13.73490
SMT-RAT SMT-RAT-final default unknown ❌ 2400.01000 2400.34000
STP stp-cryptominisat4 default unsat ✅ 308.03200 308.09400
stp-cmsat4-v15 default unsat ✅ 89.48970 89.50940
stp-cmsat4-mt-v15 default unknown ❌ 1.97703 1.97670
stp-minisat-v15 default unsat ✅ 79.89990 79.90590
Yices2 Yices default unsat ✅ 482.93000 483.09700
Z3 z3 4.4.0 default unsat ✅ 378.18400 378.29500
SMT-COMP 2016 0.10 (9/10) ABC ABC_default default_abc unknown ❌ 2400.12000 2401.45000
ABC_glucose glucose unsat ✅ 137.62200 137.69200
Boolector Boolector boolector unsat ✅ 89.58580 89.63050
Boolector preprop boolector unsat ✅ 87.18740 87.23480
CVC4 CVC4-master-2016-05-27-cfef263-main default unsat ✅ 145.39700 291.83600
MapleSTP MapleSTP default unknown ❌ 2400.02000 2401.37000
MapleSTP-mt default unsat ✅ 1896.94000 7525.64000
MathSAT mathsat-5.3.11-linux-x86_64-Main default unsat ✅ 14.31570 14.32270
MinkeyRink Minkeyrink 2016 default unsat ✅ 158.42600 560.37100
Q3B Q3B default unknown ❌ 2400.02000 7204.48000
STP stp-cms-exp-2016 default unknown ❌ 58.02120 54.91560
stp-cms-mt-2016 default unsat ✅ 77.78010 301.74800
stp-cms-st-2016 default unsat ✅ 90.87300 90.92530
stp-minisat-st-2016 default unsat ✅ 746.54600 746.96200
Yices2 Yices-2.4.2 default unsat ✅ 593.29900 593.62600
Z3 z3-4.4.1 default unsat ✅ 305.11500 305.28300
SMT-COMP 2017 0.12 (7/8) Boolector Boolector+CaDiCaL SMT17 final boolector unsat ✅ 61.95360 61.93180
Boolector SMT17 final boolector unsat ✅ 184.50400 184.47400
CVC4 CVC4-smtcomp2017-main default unsat ✅ 105.64600 211.93000
MathSAT mathsat-5.4.1-linux-x86_64-Main default unsat ✅ 10.55910 10.55760
MinkeyRink MinkeyRink 2017.3a default unsat ✅ 87.55390 170.09000
Q3B Q3B default unknown ❌ 600.09800 1799.82000
STP stp_st default unsat ✅ 59.72530 59.72370
stp_mt default unsat ✅ 71.44310 275.76600
Yices2 Yices2-Main default unsat ✅ 573.32600 573.31700
Z3 z3-4.5.0 default unsat ✅ 269.90600 269.83900
SMT-COMP 2018 Boolector Boolector_default unsat ✅ 53.62410 53.61890
CVC4 master-2018-06-10-b19c840-competition-default_default unsat ✅ 146.65500 146.64500
MathSAT mathsat-5.5.2-linux-x86_64-Main_default unsat ✅ 11.61250 11.61330
MinkeyRink Minkeyrink MT_mt unsat ✅ 99.98390 389.41700
Minkeyrink ST_st unsat ✅ 88.89570 88.84740
STP STP-CMS-st-2018_default-no-stderr unsat ✅ 94.05920 94.02110
STP-CMS-mt-2018_multicore-no-stderr unsat ✅ 93.94660 366.64200
STP-Riss-st-2018_riss-no-stderr unsat ✅ 61.41520 61.40960
Yices2 Yices 2.6.0_default unsat ✅ 511.16400 511.14000
Z3 z3-4.7.1_default unsat ✅ 464.44800 464.40500
SMT-COMP 2019 Boolector Boolector-wrapped-sq_default unsat ✅ 102.09300 102.08900
Poolector-wrapped-sq_default unsat ✅ 127.35700 504.47000
Boolector-ReasonLS boolector-ReasonLS-wrapped-sq_default unsat ✅ 145.13400 145.03000
CVC4 CVC4-2019-06-03-d350fe1-wrapped-sq_default unsat ✅ 173.22700 173.13100
MinkeyRink MinkeyRink MT-wrapped-sq_default unsat ✅ 101.41200 396.72800
MinkeyRink ST-wrapped-sq_default unsat ✅ 95.81580 95.80440
Par4 Par4-wrapped-sq_default unsat ✅ 74.72830 223.99000
STP STP-2019-wrapped-sq_default unsat ✅ 89.02280 89.01300
stp-mergesat-fixed-wrapped-sq_default unsat ✅ 94.10300 94.09940
stp-minisat-wrapped-sq_default unsat ✅ 65.77340 65.75330
stp-mt-wrapped-sq_default unsat ✅ 93.65850 364.70800
stp-portfolio-fixed-wrapped-sq_default unsat ✅ 115.86000 303.28900
stp-riss-wrapped-sq_default unknown ❌ 2.40807 2.40832
Yices2 Yices 2.6.2-wrapped-sq_default unsat ✅ 585.90300 585.70200
Yices 2.6.2 Cadical-wrapped-sq_default unsat ✅ 208.88800 208.85600
Yices 2.6.2 Cryptominisat-wrapped-sq_default unsat ✅ 173.03800 173.03500
Yices 2.6.2 MCSAT BV-wrapped-sq_default unsat ✅ 144.55100 144.52100
Yices 2.6.2 new bvsolver-wrapped-sq_default unsat ✅ 112.98600 112.98400
Z3 z3-4.8.4-d6df51951f4c-wrapped-sq_default unsat ✅ 587.47700 587.45100
SMT-COMP 2020 0.11 (8/9) Bitwuzla Bitwuzla-fixed_default unsat ✅ 70.63080 70.58560
Boolector Boolector-wrapped-sq_default unsat ✅ 80.03430 80.03100
Poolector-wrapped-sq_default unsat ✅ 102.21300 404.44000
CVC4 CVC4-sq-final_default unsat ✅ 166.70700 166.66800
LazyBV2Int LazyBV2Int20200523_default.sh unsat ✅ 595.93600 595.86100
MathSAT MathSAT5_default.sh unsat ✅ 14.21300 14.21180
MinkeyRink MinkeyRink Solver 2020.3.1_default unknown ❌ 2.16747 2.16771
MinkeyRink Solver 2020.3_default unsat ✅ 77.15780 300.32800
STP STP_default unsat ✅ 72.60670 281.52500
STP ++ Mergsat v1_default unsat ✅ 92.79740 92.79660
Yices2 Yices 2.6.2 bug fix_default unsat ✅ 92.27530 92.26790
Z3 z3-4.8.8_default unknown ❌ 1200.02000 1199.82000
SMT-COMP 2022 Bitwuzla Bitwuzla-fixed_default unsat ✅ 83.36430 83.36210
Bitwuzla-wrapped_default unsat ✅ 78.32010 78.31160
cvc5 cvc5-default-2022-07-02-b15e116-wrapped_sq unsat ✅ 162.21800 162.14900
MathSAT MathSAT-5.6.8_default unsat ✅ 16.73790 16.73600
STP STP 2022.4_default unsat ✅ 86.18790 86.18500
Yices2 Yices 2.6.2 for SMTCOMP 2021_default unsat ✅ 68.54220 68.53990
Z3++BV z3++bv_0702_default unsat ✅ 90.59040 90.58850
Z3 z3-4.8.17_default unsat ✅ 1175.48000 1175.23000
SMT-COMP 2023 0.17 (5/6) Bitwuzla Bitwuzla-fixed_default unsat ✅ 138.56000 138.48700
cvc5 cvc5-default-2023-05-16-ea045f305_sq unsat ✅ 188.74700 188.74900
STP STP 2022.4_default unsat ✅ 77.65190 77.63950
STP 2022.4_default unsat ✅ 77.33790 77.34200
UltimateEliminator UltimateIntBlastingWrapper+SMTInterpol_default unknown ❌ 1200.07000 1264.75000
Yices2 Yices 2 for SMTCOMP 2023_default unsat ✅ 774.42700 774.40200
Z3-Owl z3-Owl-Final_default unsat ✅ 158.55600 158.52600
z3-Owl-Final_default unsat ✅ 168.27000 168.22000