Benchmark

non-incremental/QF_BV/spear/openldap_v2.3.35/servers_slapd_a_vc149922.smt2 

Benchmarks generated by the Calysto static checker.  Provided by Domagoj
Babic.  See http://www.cs.ubc.ca/~babic/index_benchmarks.htm.  Translated using
Spear Sf2Smt translator.
Benchmark
Size75470
Compressed Size11870
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2007-07-03
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status sat
Inferred Status sat
Size 75462
Compressed Size11886
Max. Term Depth5
Asserts 694
Declared Functions0
Declared Constants730
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

ite10 =>3 =724 distinct2
concat131 extract233 bvand50 bvor74
bvxor126 bvadd247 bvsub1 bvult2
bvule5 bvshl64 bvlshr67 sign_extend27

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT-COMP 2008 0.57 (3/7) Beaver Beaver-1.0 unknown ❌ 1800.80000 —
Boolector Boolector sat ✅ 1.70000 —
CVC3 CVC3-1.5 unknown ❌ 1801.00000 —
MathSAT MathSAT-4.2 sat ✅ 940.00000 —
Spear Spear unknown ❌ 1212.10000 —
Spear v1.9 (fh-1-2) unknown ❌ 1800.80000 —
SWORD SWORD v0.2 sat ✅ 131.20000 —
Z3 Z3.2 unknown ❌ 1800.80000 —
SMT-COMP 2010 1.00 (0/3) CVC3 CVC3 2.3 unknown ❌ 1200.10000 —
SONOLAR SONOLAR r252 unknown ❌ 1200.00000 —
STP simplifyingSTP unknown ❌ 1200.00000 —
SMT-COMP 2012 0.29 (5/7) 4Simp 4Simp sat ✅ 429.80000 —
Abziz AbzizPortfolio+BOOLECTOR+MATHSAT+SONOLAR+STP2+Z3_6 sat ✅ 6.60000 —
Boolector Boolector sat ✅ 0.30000 —
CVC3 CVC3 v2.4.2 unknown ❌ 987.30000 —
CVC4 1.0rc.3931 unknown ❌ 1200.00000 —
MathSAT MathSAT-HeavyBV sat ✅ 3.70000 —
SONOLAR SONOLAR sat ✅ 1158.30000 —
Z3 Z3 unknown ❌ 1200.00000 —
SMT-COMP 2011 0.83 (1/6) Boolector Boolector 1.5.23-833 unknown ❌ 1200.00000 —
CVC3 CVC3 v2.4 unknown ❌ 1200.20000 —
MathSAT MathSAT5 sat ✅ 1.90000 —
SONOLAR SONOLAR unknown ❌ 1200.00000 —
STP simplifyingSTP unknown ❌ 1200.00000 —
STP2 unknown ❌ 1200.00000 —
Z3 Z3 unknown ❌ 1200.00000 —
SMT Evaluation 2013 0.33 (6/9) 4Simp 4Simp-SMT-COMP-2012 default sat ✅ 365.12400 —
Abziz abziz_portfolio_2011_minfeatures_2 default sat ✅ 6.15730 —
abziz_portfolio_2011_solvers_2 default sat ✅ 5.85188 —
abziz_portfolio_2012_minfeatures default sat ✅ 370.00300 —
abziz_portfolio_2012_solvers default sat ✅ 5.31668 —
AbzizPortfolio-SMT-COMP-2012 default sat ✅ 6.15683 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default unknown ❌ — —
Boolector-SMT-COMP-2011 default sat ✅ 1208.41000 —
Boolector-SMT-COMP-2012 default sat ✅ 0.20439 —
CVC3 CVC3-SMT-COMP-2010 default unknown ❌ — —
CVC3-SMT-COMP-2011 default unknown ❌ — —
CVC3-SMT-COMP-2012 default unknown ❌ — —
CVC4 CVC4-SMT-COMP-2012-Resubmission default unknown ❌ — —
CVC4-SMT-EVAL-2013 default unknown ❌ — —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default sat ✅ 130.90700 —
MathSAT5-HeavyBV-SMT-COMP-2012 default sat ✅ 1.88010 —
MathSAT5-SMT-COMP-2011 default sat ✅ 0.74203 —
MathSAT5-SMT-COMP-2012 default sat ✅ 0.96097 —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default sat ✅ 1184.46000 —
SONOLAR-SMT-COMP-2010 default sat ✅ 6.54401 —
SONOLAR-SMT-COMP-2011 default unknown ❌ — —
SONOLAR-SMT-COMP-2012 default sat ✅ 1190.36000 —
STP STP2-SMT-COMP-2011 default sat ✅ 1.03548 —
STP2-SMT-COMP-2012 default sat ✅ 247.00800 —
simplifyingSTP-SMT-COMP-2010 default unknown ❌ — —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default unknown ❌ — —
Z3-SMT-COMP-2011 default unknown ❌ — —
SMT-COMP 2014 0.22 (7/9) 4Simp 4Simp - 2014 default sat ✅ 1220.21000 1220.81000
Abziz abziz_portfolio_all_features default sat ✅ 488.10300 488.17600
abziz_portfolio_min_features default sat ✅ 489.30100 489.37300
Boolector Boolector boolector sat ✅ 383.19200 383.33300
CVC4 CVC4 f7118b2 default unknown ❌ 1197.37000 2402.20000
MathSAT MathSAT-5.2.12-Main default sat ✅ 652.66300 652.83000
SONOLAR sonolar_smtcomp-2014 default sat ✅ 1954.87000 1955.66000
STP stp-cryptominisat4 default sat ✅ 0.89935 0.89086
Yices2 Yices-2.2.1-smtcomp2014 default sat ✅ 11.83250 11.82220
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-COMP-2014 default unknown ❌ 2399.12000 2400.02000
SMT-COMP 2015 0.57 (3/7) Boolector Boolector SMT15 QF_BV final boolector_qf_bv unknown ❌ 2400.01000 2401.14000
CVC4 CVC4-master-2015-06-15-9b32405-main default unknown ❌ 2400.20000 4824.21000
CVC4-experimental-2015-06-15-ff5745a-main default unknown ❌ 2400.03000 4819.43000
MathSAT MathSat 5.3.6 main smtcomp2015_main sat ✅ 8.00419 8.00678
SMT-RAT SMT-RAT-final default unknown ❌ 2400.01000 2400.89000
STP stp-cryptominisat4 default sat ✅ 0.87754 0.87687
stp-cmsat4-v15 default sat ✅ 1.11406 1.11383
stp-cmsat4-mt-v15 default unknown ❌ 0.74216 0.74089
stp-minisat-v15 default sat ✅ 1.06876 1.06784
Yices2 Yices default sat ✅ 14.81160 14.81670
Z3 z3 4.4.0 default unknown ❌ 2400.01000 2400.91000
SMT-COMP 2016 0.50 (5/10) ABC ABC_default default_abc unknown ❌ 82.79760 80.36640
ABC_glucose glucose sat ✅ 1199.52000 1200.16000
Boolector Boolector boolector unknown ❌ 2400.03000 2401.39000
Boolector preprop boolector unknown ❌ 2400.02000 2401.44000
CVC4 CVC4-master-2016-05-27-cfef263-main default unknown ❌ 2400.03000 4824.11000
MapleSTP MapleSTP default unknown ❌ 2400.02000 2401.19000
MapleSTP-mt default unknown ❌ 2400.06000 9538.19000
MathSAT mathsat-5.3.11-linux-x86_64-Main default sat ✅ 225.26700 225.32200
MinkeyRink Minkeyrink 2016 default sat ✅ 638.82700 2487.92000
Q3B Q3B default unknown ❌ 2400.10000 7201.71000
STP stp-cms-exp-2016 default sat ✅ 469.91800 470.21000
stp-cms-mt-2016 default sat ✅ 246.35200 976.04300
stp-cms-st-2016 default sat ✅ 259.10800 259.23200
stp-minisat-st-2016 default unknown ❌ 2400.10000 2401.45000
Yices2 Yices-2.4.2 default sat ✅ 10.45240 10.45830
Z3 z3-4.4.1 default unknown ❌ 2400.01000 2401.36000
SMT-COMP 2017 0.50 (4/8) Boolector Boolector+CaDiCaL SMT17 final boolector sat ✅ 0.07366 0.07332
Boolector SMT17 final boolector sat ✅ 0.07277 0.07182
CVC4 CVC4-smtcomp2017-main default sat ✅ 18.59770 37.19750
MathSAT mathsat-5.4.1-linux-x86_64-Main default sat ✅ 75.75140 75.74420
MinkeyRink MinkeyRink 2017.3a default unknown ❌ 600.06500 1207.85000
Q3B Q3B default unknown ❌ 600.01400 1799.51000
STP stp_st default unknown ❌ 600.10500 600.01700
stp_mt default unknown ❌ 600.06500 1911.61000
Yices2 Yices2-Main default sat ✅ 0.31837 0.31636
Z3 z3-4.5.0 default unknown ❌ 600.02000 599.96800
SMT-COMP 2018 0.43 (4/7) Boolector Boolector_default sat ✅ 0.03589 0.03584
CVC4 master-2018-06-10-b19c840-competition-default_default unknown ❌ 1200.10000 1200.06000
MathSAT mathsat-5.5.2-linux-x86_64-Main_default sat ✅ 499.16100 499.00200
MinkeyRink Minkeyrink MT_mt unknown ❌ 1200.11000 4771.53000
Minkeyrink ST_st unknown ❌ 1200.02000 1199.97000
STP STP-CMS-st-2018_default-no-stderr sat ✅ 251.21400 251.21200
STP-CMS-mt-2018_multicore-no-stderr sat ✅ 389.80500 1548.45000
STP-Riss-st-2018_riss-no-stderr sat ✅ 703.79300 703.73100
Yices2 Yices 2.6.0_default sat ✅ 19.38540 19.38230
Z3 z3-4.7.1_default unknown ❌ 1200.03000 1199.95000
SMT-COMP 2020 0.22 (7/9) Bitwuzla Bitwuzla-fixed_default sat ✅ 2.29920 2.29900
Boolector Boolector-wrapped-sq_default sat ✅ 0.09616 0.09614
Poolector-wrapped-sq_default sat ✅ 0.17792 0.19873
CVC4 CVC4-sq-final_default sat ✅ 1024.10000 1023.69000
LazyBV2Int LazyBV2Int20200523_default.sh unknown ❌ 1200.03000 1199.96000
MathSAT MathSAT5_default.sh sat ✅ 0.99824 0.99809
MinkeyRink MinkeyRink Solver 2020.3.1_default unknown ❌ 0.83695 0.83720
MinkeyRink Solver 2020.3_default sat ✅ 71.81130 282.42000
STP STP_default sat ✅ 68.65220 270.52000
STP ++ Mergsat v1_default sat ✅ 94.24250 94.23840
Yices2 Yices 2.6.2 bug fix_default sat ✅ 168.24500 168.20400
Z3 z3-4.8.8_default unknown ❌ 1200.03000 1199.91000
SMT-COMP 2021 0.25 (3/4) Bitwuzla Bitwuzla-fixed_default sat ✅ 2.35900 2.35857
MathSAT mathsat-5.6.6_default sat ✅ 24.83410 24.83040
STP STP 2021.0_default sat ✅ 103.40600 382.91800
Z3 z3-4.8.11_default unknown ❌ 1200.01000 1199.64000
SMT-COMP 2023 0.33 (4/6) Bitwuzla Bitwuzla-fixed_default sat ✅ 3.21564 3.21536
cvc5 cvc5-default-2023-05-16-ea045f305_sq sat ✅ 407.77800 407.72000
STP STP 2022.4_default sat ✅ 86.00150 86.00850
STP 2022.4_default sat ✅ 85.97140 85.96870
UltimateEliminator UltimateIntBlastingWrapper+SMTInterpol_default unknown ❌ 1200.03000 1265.81000
Yices2 Yices 2 for SMTCOMP 2023_default sat ✅ 0.30318 0.30314
Z3-Owl z3-Owl-Final_default unknown ❌ 1200.10000 1199.82000
z3-Owl-Final_default unknown ❌ 1200.03000 1199.67000
SMT-COMP 2024 0.17 (5/6) Bitwuzla Bitwuzla sat ✅ 18.89952 18.79906
cvc5 cvc5 sat ✅ 154.58836 154.35493
SMTInterpol SMTInterpol unknown ❌ 1201.71941 1240.27489
STP STP sat ✅ 1.63720 1.53737
Yices2 Yices2 sat ✅ 25.85895 25.75842
Z3alpha Z3-alpha sat ✅ 494.19007 493.97852
SMT-COMP 2025 0.33 (6/9) Bitwuzla Bitwuzla sat ✅ 14.13500 14.00645
Bitwuzla-MachBV-base sat ✅ 8.03935 7.91687
Bitwuzla-MachBV Bitwuzla-MachBV sat ✅ 13.95592 13.81991
BVDecide bv_decide sat ✅ 91.95259 91.84233
bv_decide-nokernel sat ✅ 90.27018 90.17343
cvc5 cvc5 sat ✅ 176.89810 176.73536
SMTInterpol SMTInterpol unknown ❌ 1201.68955 1230.26497
Yices2 Yices2 sat ✅ 65.64564 65.52088
Z3alpha Z3-alpha sat ✅ 790.78923 3158.03815
Z3 Z3-alpha-base unknown ❌ 1201.34691 1201.05361
Z3-Owl-base unknown ❌ 1201.42101 1201.09815
z3siri-base unknown ❌ 1201.34680 1201.05500
Z3-Owl Z3-Owl unknown ❌ 1201.76219 1201.19546