Benchmark

non-incremental/QF_BV/tacas07/Y86_std.smt2

This directory contains benchmarks used in the experimental results
reported in the following paper:

Title: Deciding Bit-Vector Arithmetic with Abstraction
Authors: R. E. Bryant, D. Kroening, J. Ouaknine, S. A. Seshia, O. Strichman, B. Brady

If you use these benchmarks, we will appreciate hearing from you by 
e-mail to sseshia@eecs.berkeley.edu. Additionally, the following sources 
must be acknowledged:

1) Y86_std.smt, Y86_btnft.smt: 
   Generated from a UCLID model created by R. E. Bryant, as described
   in the following paper:
    http://www.cs.cmu.edu/~bryant/pubdir/CMU-CS-05-195.pdf

2) s-40-50-bv.smt: 
   Generated by V. Ganapathy for finding format-string exploits in
   the WiSA project (http://www.cs.wisc.edu/wisa)
 
3) BBB-32.smt, rfunit_flat-64.smt:
   Generated from benchmarks generated by Z. Hanna of Intel.

All benchmarks are in the SMT-LIB format, as described at:
http://combination.cs.uiowa.edu/smtlib/

Benchmark
Size	182881
Compressed Size	27842
License	Creative Commons Attribution 4.0 International (CC-BY-4.0)
Category	industrial
First Occurrence	2007-07-03
Generated By	—
Generated On	—
Generator	—
Dolmen OK	1
strict Dolmen OK	1
check-sat calls	1

Query 1

Status	unsat
Inferred Status	unsat
Size	182873
Compressed Size	27446
Max. Term Depth	2009
Asserts	1
Declared Functions	0
Declared Constants	246
Declared Sorts	0
Defined Functions	0
Defined Recursive Functions	0
Defined Sorts	0
Constants	0
Declared Datatypes	0

Symbols

ite	733	not	1703	or	2334	and	2542
=	3939	let	326	extract	12	bvneg	1
bvadd	78

Evaluations

Evaluation	Rating	Solver	Variant	Result	Wallclock	CPU Time
SMT-COMP 2008	0.14 (6/7)	Beaver	Beaver-1.0	unsat ✅	1746.40000	—
		Boolector	Boolector	unsat ✅	335.70000	—
		CVC3	CVC3-1.5	unsat ✅	1461.60000	—
		MathSAT	MathSAT-4.2	unsat ✅	62.40000	—
		Spear	Spear	unsat ✅	599.50000	—
			Spear v1.9 (fh-1-2)	unknown ❌	1800.80000	—
		SWORD	SWORD v0.2	unknown ❌	1800.80000	—
		Z3	Z3.2	unsat ✅	29.30000	—
SMT-COMP 2010	0.33 (2/3)	CVC3	CVC3 2.3	unknown ❌	1200.00000	—
		SONOLAR	SONOLAR r252	unsat ✅	1168.40000	—
		STP	simplifyingSTP	unsat ✅	702.90000	—
SMT-COMP 2012	0.29 (5/7)	4Simp	4Simp	unsat ✅	95.70000	—
		Abziz	AbzizPortfolio+BOOLECTOR+MATHSAT+SONOLAR+STP2+Z3_6	unknown ❌	1200.00000	—
		Boolector	Boolector	unsat ✅	269.40000	—
		CVC3	CVC3 v2.4.2	unsat ✅	893.60000	—
			CVC4 1.0rc.3931	unsat ✅	38.20000	—
		MathSAT	MathSAT-HeavyBV	unknown ❌	1200.20000	—
		SONOLAR	SONOLAR	unsat ✅	206.10000	—
		Z3	Z3	unsat ✅	1084.30000	—
SMT-COMP 2011		Boolector	Boolector 1.5.23-833	unsat ✅	202.70000	—
		CVC3	CVC3 v2.4	unsat ✅	535.10000	—
		MathSAT	MathSAT5	unsat ✅	24.80000	—
		SONOLAR	SONOLAR	unsat ✅	200.20000	—
		STP	simplifyingSTP	unsat ✅	750.20000	—
			STP2	unsat ✅	1162.60000	—
		Z3	Z3	unsat ✅	1167.20000	—
SMT Evaluation 2013		4Simp	4Simp-SMT-COMP-2012 default	unsat ✅	39.39490	—
		Abziz	abziz_portfolio_2011_minfeatures_2 default	unsat ✅	104.00800	—
			abziz_portfolio_2011_solvers_2 default	unsat ✅	15.22290	—
			abziz_portfolio_2012_minfeatures default	unsat ✅	16.36030	—
			abziz_portfolio_2012_solvers default	unsat ✅	16.43590	—
			AbzizPortfolio-SMT-COMP-2012 default	unsat ✅	15.19230	—
		Boolector	Boolector-1.5.118-SMT-EVAL-2013 default	unsat ✅	87.05320	—
			Boolector-SMT-COMP-2011 default	unsat ✅	98.89140	—
			Boolector-SMT-COMP-2012 default	unsat ✅	134.35400	—
		CVC3	CVC3-SMT-COMP-2010 default	unsat ✅	727.18100	—
			CVC3-SMT-COMP-2011 default	unsat ✅	636.33200	—
			CVC3-SMT-COMP-2012 default	unsat ✅	654.89900	—
		CVC4	CVC4-SMT-COMP-2012-Resubmission default	unsat ✅	49.69080	—
			CVC4-SMT-EVAL-2013 default	unsat ✅	27.12970	—
		MathSAT	MathSAT5-5.2.6-SMT-EVAL-2013 default	unsat ✅	8.31606	—
			MathSAT5-HeavyBV-SMT-COMP-2012 default	unknown ❌	—	—
			MathSAT5-SMT-COMP-2011 default	unsat ✅	9.98506	—
			MathSAT5-SMT-COMP-2012 default	unsat ✅	11.11640	—
		SONOLAR	SONOLAR-2013-05-15-SMT-EVAL-2013 default	unsat ✅	58.30650	—
			SONOLAR-SMT-COMP-2010 default	unsat ✅	476.15600	—
			SONOLAR-SMT-COMP-2011 default	unsat ✅	95.67010	—
			SONOLAR-SMT-COMP-2012 default	unsat ✅	74.47590	—
		STP	STP2-SMT-COMP-2011 default	unsat ✅	72.73030	—
			STP2-SMT-COMP-2012 default	unsat ✅	44.42730	—
			simplifyingSTP-SMT-COMP-2010 default	unsat ✅	943.24900	—
		Z3	Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default	unsat ✅	196.66000	—
			Z3-SMT-COMP-2011 default	unsat ✅	307.56900	—
SMT-COMP 2014		4Simp	4Simp - 2014 default	unsat ✅	48.25990	48.26970
		Abziz	abziz_portfolio_all_features default	unsat ✅	47.66110	47.51750
			abziz_portfolio_min_features default	unsat ✅	35.20670	65.18240
		Boolector	Boolector boolector	unsat ✅	103.59800	103.61100
		CVC4	CVC4 f7118b2 default	unsat ✅	70.63130	141.61600
		MathSAT	MathSAT-5.2.12-Main default	unsat ✅	7.78667	7.78881
		SONOLAR	sonolar_smtcomp-2014 default	unsat ✅	50.76340	50.77330
		STP	stp-cryptominisat4 default	unsat ✅	285.19300	285.25400
		Yices2	Yices-2.2.1-smtcomp2014 default	unsat ✅	482.65700	482.84000
		Z3	Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-COMP-2014 default	unsat ✅	392.73300	392.85300
SMT-COMP 2015	0.14 (6/7)	Boolector	Boolector SMT15 QF_BV final boolector_qf_bv	unsat ✅	100.19300	100.21400
		CVC4	CVC4-master-2015-06-15-9b32405-main default	unsat ✅	76.02740	152.54500
			CVC4-experimental-2015-06-15-ff5745a-main default	unsat ✅	75.68410	151.83500
		MathSAT	MathSat 5.3.6 main smtcomp2015_main	unsat ✅	9.68428	9.68753
		SMT-RAT	SMT-RAT-final default	unknown ❌	2400.08000	2400.91000
		STP	stp-cryptominisat4 default	unsat ✅	317.71500	317.84500
			stp-cmsat4-v15 default	unsat ✅	88.40220	88.43460
			stp-cmsat4-mt-v15 default	unknown ❌	1.93068	1.92971
			stp-minisat-v15 default	unsat ✅	82.49450	82.52850
		Yices2	Yices default	unsat ✅	403.77000	403.86100
		Z3	z3 4.4.0 default	unsat ✅	257.87400	257.93900
SMT-COMP 2016	0.10 (9/10)	ABC	ABC_default default_abc	unknown ❌	2400.03000	2401.38000
			ABC_glucose glucose	unsat ✅	148.82400	148.24700
		Boolector	Boolector boolector	unsat ✅	107.39700	107.44900
			Boolector preprop boolector	unsat ✅	111.10300	111.16500
		CVC4	CVC4-master-2016-05-27-cfef263-main default	unsat ✅	96.27190	193.16000
		MapleSTP	MapleSTP default	unknown ❌	2400.03000	2401.20000
			MapleSTP-mt default	unsat ✅	2098.97000	8333.81000
		MathSAT	mathsat-5.3.11-linux-x86_64-Main default	unsat ✅	9.52771	9.53285
		MinkeyRink	Minkeyrink 2016 default	unsat ✅	149.80300	527.07700
		Q3B	Q3B default	unknown ❌	2400.10000	7203.99000
		STP	stp-cms-exp-2016 default	unknown ❌	97.65420	94.07090
			stp-cms-mt-2016 default	unsat ✅	83.15770	321.45100
			stp-cms-st-2016 default	unsat ✅	67.93940	67.96770
			stp-minisat-st-2016 default	unknown ❌	2400.02000	2401.41000
		Yices2	Yices-2.4.2 default	unsat ✅	519.00600	519.28700
		Z3	z3-4.4.1 default	unsat ✅	241.11300	241.22800
SMT-COMP 2017	0.12 (7/8)	Boolector	Boolector+CaDiCaL SMT17 final boolector	unsat ✅	52.59520	52.59320
			Boolector SMT17 final boolector	unsat ✅	191.48300	191.46700
		CVC4	CVC4-smtcomp2017-main default	unsat ✅	88.00480	176.59100
		MathSAT	mathsat-5.4.1-linux-x86_64-Main default	unsat ✅	8.43285	8.43096
		MinkeyRink	MinkeyRink 2017.3a default	unsat ✅	64.92740	124.22000
		Q3B	Q3B default	unknown ❌	600.08400	1799.69000
		STP	stp_st default	unsat ✅	56.07950	56.05000
			stp_mt default	unsat ✅	72.36170	279.99400
		Yices2	Yices2-Main default	unsat ✅	514.93200	514.87300
		Z3	z3-4.5.0 default	unsat ✅	276.35800	276.27400
SMT-COMP 2018		Boolector	Boolector_default	unsat ✅	54.27550	54.27260
		CVC4	master-2018-06-10-b19c840-competition-default_default	unsat ✅	107.79700	107.69000
		MathSAT	mathsat-5.5.2-linux-x86_64-Main_default	unsat ✅	11.63320	11.63250
		MinkeyRink	Minkeyrink MT_mt	unsat ✅	96.16710	374.58700
			Minkeyrink ST_st	unsat ✅	91.73400	91.72120
		STP	STP-CMS-st-2018_default-no-stderr	unsat ✅	92.71700	92.69910
			STP-CMS-mt-2018_multicore-no-stderr	unsat ✅	89.78020	350.25400
			STP-Riss-st-2018_riss-no-stderr	unsat ✅	67.34000	67.34190
		Yices2	Yices 2.6.0_default	unsat ✅	450.64400	450.64100
		Z3	z3-4.7.1_default	unsat ✅	329.49400	329.49300
SMT-COMP 2020	0.11 (8/9)	Bitwuzla	Bitwuzla-fixed_default	unsat ✅	97.75480	97.73200
		Boolector	Boolector-wrapped-sq_default	unsat ✅	162.22700	162.17100
			Poolector-wrapped-sq_default	unsat ✅	185.39400	733.71100
		CVC4	CVC4-sq-final_default	unsat ✅	141.89500	141.88800
		LazyBV2Int	LazyBV2Int20200523_default.sh	unsat ✅	562.40800	562.16100
		MathSAT	MathSAT5_default.sh	unsat ✅	10.18770	10.18670
		MinkeyRink	MinkeyRink Solver 2020.3.1_default	unknown ❌	2.09360	2.09386
			MinkeyRink Solver 2020.3_default	unsat ✅	74.15860	287.33000
		STP	STP_default	unsat ✅	69.37730	268.84800
			STP ++ Mergsat v1_default	unsat ✅	91.72780	91.72620
		Yices2	Yices 2.6.2 bug fix_default	unsat ✅	67.70100	67.70200
		Z3	z3-4.8.8_default	unknown ❌	1200.11000	1200.04000
SMT-COMP 2021		Bitwuzla	Bitwuzla-fixed_default	unsat ✅	79.46480	79.45640
		MathSAT	mathsat-5.6.6_default	unsat ✅	10.96600	10.96590
		STP	STP 2021.0_default	unsat ✅	69.94520	268.09100
		Z3	z3-4.8.11_default	unsat ✅	662.22800	662.21100
SMT-COMP 2025		Bitwuzla	Bitwuzla	unsat ✅	44.84181	44.70286
			Bitwuzla-MachBV-base	unsat ✅	51.67196	51.53738
		Bitwuzla-MachBV	Bitwuzla-MachBV	unsat ✅	64.69504	64.56967
		BVDecide	bv_decide	unknown ❌	1201.47248	1201.15744
			bv_decide-nokernel	unsat ✅	784.20915	784.01740
		cvc5	cvc5	unsat ✅	107.97561	107.83059
		SMTInterpol	SMTInterpol	unsat ✅	15.51246	35.73761
		Yices2	Yices2	unsat ✅	90.54742	90.41368
		Z3alpha	Z3-alpha	unsat ✅	133.39673	531.52614
		Z3	Z3-alpha-base	unsat ✅	433.68784	433.53217
			Z3-Owl-base	unknown ❌	1201.29783	1201.07345
			z3siri-base	unsat ✅	420.43632	420.25624
		Z3-Owl	Z3-Owl	unsat ✅	78.33780	78.21035