PENLABSTRESSTEST runs a series of tests on the solver.
This is to be used as an automated tool to run all possible
test of the solver (e.g., a whole benchmarking library) to check
how a possible source change influenced the behaviour of
the solver.
The problems to test are generated by FEEDER. This routine executes them
and prints out one-liner statistics for comparison.
INPUT:
feeder - routine returning the penlab objects to be ran,
optional, if not present, the default feeder is used;
it should look like:
function [status,name,prob,res] = feeder(no)
and to test number no=1..maxtests return status=0,
problem name and problem as penlab object,
expected results may be present in res, otherwise [],
if there are problems with the test, status<0,
status=1 means out of range of test
timestats - if (timestats), timing (tic/toc) is printed in the summary,
optional, default true
logs - if (logs), logs are produced for each of the tests in
the current directory; optional, default false
EXAMPLES:
% use Matlab's diary to record the summary output
diary('logtest_summary.txt');
penlabstresstest();
diary off;
% using your own test set and default feeder:
mytestset = {{ 'sdpa', 'datafiles/truss1.dat-s', -8.999996e+00 }, ...
{ 'bmi', 'datafiles/bmi_example.mat' }};
myfeeder = @(no) penlabtestfeeder(no,mytestset);
penlabstresstest(myfeeder);
% default feeder but turn off timing and generate logs
penlabstresstest(@penlabtestfeeder,0,1);
See also penlabtestfeeder0001 function [] = penlabstresstest(feeder,timestats,logs) 0002 % PENLABSTRESSTEST runs a series of tests on the solver. 0003 % 0004 % This is to be used as an automated tool to run all possible 0005 % test of the solver (e.g., a whole benchmarking library) to check 0006 % how a possible source change influenced the behaviour of 0007 % the solver. 0008 % 0009 % The problems to test are generated by FEEDER. This routine executes them 0010 % and prints out one-liner statistics for comparison. 0011 % 0012 % INPUT: 0013 % feeder - routine returning the penlab objects to be ran, 0014 % optional, if not present, the default feeder is used; 0015 % it should look like: 0016 % function [status,name,prob,res] = feeder(no) 0017 % and to test number no=1..maxtests return status=0, 0018 % problem name and problem as penlab object, 0019 % expected results may be present in res, otherwise [], 0020 % if there are problems with the test, status<0, 0021 % status=1 means out of range of test 0022 % timestats - if (timestats), timing (tic/toc) is printed in the summary, 0023 % optional, default true 0024 % logs - if (logs), logs are produced for each of the tests in 0025 % the current directory; optional, default false 0026 % 0027 % EXAMPLES: 0028 % % use Matlab's diary to record the summary output 0029 % diary('logtest_summary.txt'); 0030 % penlabstresstest(); 0031 % diary off; 0032 % 0033 % % using your own test set and default feeder: 0034 % mytestset = {{ 'sdpa', 'datafiles/truss1.dat-s', -8.999996e+00 }, ... 0035 % { 'bmi', 'datafiles/bmi_example.mat' }}; 0036 % myfeeder = @(no) penlabtestfeeder(no,mytestset); 0037 % penlabstresstest(myfeeder); 0038 % 0039 % % default feeder but turn off timing and generate logs 0040 % penlabstresstest(@penlabtestfeeder,0,1); 0041 % 0042 % 0043 % See also penlabtestfeeder 0044 % 0045 0046 % This file is a part of PENLAB package distributed under GPLv3 license 0047 % Copyright (c) 2013 by J. Fiala, M. Kocvara, M. Stingl 0048 % Last Modified: 5/12/2013 0049 0050 % relative precision for comparison the objective? 0051 tol = 1e-4; 0052 0053 if (nargin<3) 0054 logs = 0; 0055 end 0056 0057 if (nargin<2) 0058 timestats = 1; 0059 end 0060 0061 if (nargin<1) 0062 % use default feeder with the default test set 0063 feeder = @penlabtestfeeder; 0064 end 0065 0066 % run each test feeder returns 0067 no = 1; 0068 nowrong = 0; 0069 while (1) 0070 0071 try 0072 [status,name,prob,res] = feeder(no); 0073 catch 0074 status = -99; 0075 name = 'FEEDERPROBLEM'; 0076 prob = []; 0077 res = []; 0078 end 0079 0080 if (status==1) 0081 % no more tests 0082 break; 0083 elseif (status==0) 0084 % all OK, run test 0085 0086 try 0087 % set common option settings: 0088 % turn off printing to the screen 0089 prob.opts.outlev = 0; 0090 if (logs) 0091 c = clock; 0092 stamp = sprintf('%04i%02i%02i-%02i%02i',c(1:5)); 0093 prob.opts.outlev_file = 2; 0094 prob.opts.out_filename = ['logtest-' name '-' stamp '.txt']; 0095 else 0096 prob.opts.outlev_file = 0; 0097 end 0098 0099 ifail = prob.solve(); 0100 0101 if (~isempty(res)) 0102 if (abs(prob.objx - res)<tol*(abs(prob.objx)+abs(res))) 0103 cmp = 'ok'; 0104 else 0105 cmp = 'XX'; 0106 end 0107 else 0108 cmp = ' '; 0109 end 0110 if (timestats) 0111 fprintf('%-20s: %10.1fs %4i/%4i/%4i %10.2e %2s %2i\n',name,... 0112 prob.stats_time_total,prob.miter,prob.initer,prob.lsiter,... 0113 prob.objx,cmp,ifail); 0114 else 0115 fprintf('%-20s: %4i/%4i/%4i %10.2e %2s %2i\n',name,... 0116 prob.miter,prob.initer,prob.lsiter,... 0117 prob.objx,cmp,ifail); 0118 end 0119 catch 0120 fprintf('%-20s: SOLVER HARD ERROR!\n',name); 0121 end 0122 0123 clear prob; 0124 0125 else 0126 % problem with generating the test 0127 fprintf('%-20s: cannot generate (%i)\n',name,status); 0128 nowrong = nowrong + 1; 0129 if (nowrong>10) 0130 break; 0131 end 0132 end 0133 0134 no = no + 1; 0135 0136 end 0137 0138 fprintf('END OF TESTS\n'); 0139 0140 end 0141