Jo˜aoW.Cangussu
DepartmentofComputerScienceUniversityofTexasatDallasRichardison-TX75083-0688,USA
cangussu@utdallas.edu
AdityaP.Mathur
DepartmentofComputerSciences
PurdueUniversity
WestLafayette-IN47907-1398,USA
apm@cs.purdue.edu
RaymondA.DeCarloDepartmentofElectricalandComputerEngineeringPurdueUniversity
WestLafayette-IN47907-1285,USA
decarlo@ecn.purdue.edu
Abstract
Wereportastudytodeterminetheimpactoffourtypesofdisturbancesonthefailureintensityofasoftwareproductundergoingsystemtest.Hardwarefailures,discoveryofacriticalfault,attritioninthetestteam,areexamplesofdisturbancesthatwilllikelyaffecttheconvergenceofthefailureintensitytoitsdesiredvalue.Suchdisturbancesaremodeledasimpulse,pulse,step,andwhitenoise.Ourstudyexamined,inquantitativeterms,theimpactofsuchdistur-bancesontheconvergencebehaviorofthefailureinten-sity.Resultsfromthisstudyrevealthatthebehaviorofthestatemodel,proposedelsewhere,isconsistentwithwhatonemightpredict.Themodelisusefulinthatitprovidesaquantitativemeasureofthedelayonecanexpectwhenadisturbanceoccurs.
1Introduction
Thesystemtestphaseofthesoftwarelifecycleissub-jecttovarioustypesofdisturbances,bothunforeseenandknownapriori.Whenadisturbanceoccursduringthetestphase,itmaycauseadelayintherealizationoftheobjec-tives.Inthestudyreportedhere,thefailureintensityofthesoftwareproductundertestisofinteresttous.Thuswewanttounderstand,andpredict,howwilltheconvergenceoffailureintensitytoitsdesiredvaluebeaffectedduetoadisturbance.Thisstudyisbasedentirelyonamodelofthesoftwaretestprocessproposedelsewhere[1].
tothetwoquestionsabove.Withrespecttothisstatemodel,ourinterestisinobtainingananswertothefollowingtwoquestions.
Disturbancemodeling:Howdoesonemodelquantitativelythedisturbancesthatcanaffecttheconvergenceofthetestprocesstotheob-jective?
Impactofdisturbances:Howdoesadistur-banceeffecttheconvergenceofthetestpro-cesstowardsthefailureintensityobjective?
1.2Problemcontext
TheuseoffeedbackcontrolduringtheSTPisillustratedin
Figure1.Thefigureshowsfourkeycomponentsthatpar-ticipateinthecontrolprocess.ThesearetheActualSTPwhichconsistsofthetestengineers,tools,documentation,etc.,aStateModeloftheSTPwhichisModelS,aController,andthetestmanager.Themanage-mentdecideshowmanytesterstoemploytotesttheprod-uct.Thisnumberistheinitialvalueof.Thetestmanagerestimatesthequalityofthetestprocess.Thecomplexityofthesoftwareundertestiscomputedasaconvexcombina-tionofseveralwellknowncomplexitymetricssuchasthenumberoffunctionpoints,linesofcode,andthenumberofdataflows[3].Aninitialestimateofthefailureinten-sityismadeofthesoftwareproductreadytoenterthetestphase.Furthermore,weassumethattoplanandmonitortheprogressoftheSTP,thetestmanagerdi-videstheentirephaseintoasequenceofcheckpointsdenotedbycpcpcpwithcpbeingthefirstcheck-pointaftertestinghasbegunandcpthedeadline.There-alizationofthefailureintensityobjectiveisdistributedoverthesecheckpoints.Thus,checkpointcpisspecifiedasa
pair(
),whereissometimepriortothedeadlineandisthedesiredfailureintensityoftheproductatcheck-pointcp.
Atcheckpointcp,oftheproductisestimatedandcomparedagainsttheexpectedcomputedbythestatemodel.Theerrorsignalisinputtoacon-troller.Usingthiserrorsignal,,,and,thecontroller
computesasetofpossiblechanges
andthatcouldbemadetoand,respectively,inorderfortheSTPtomeetthereliabilityobjectiveonorbeforethedeadline.Thecomputedchangesaremadeavailabletothetestmanagerwhomayormaynotchoosetoignorethem.ThoughFig-ure1givestheimpressionthatonlyasinglepair(
and)ofvaluesisoutputbythecontroller,inrealitythecon-trolleroutputsafinitesetofsuchpairsfromwhichthetestmanagercouldselect.TheSTPresumeswithaworkforce
of(
)andaprocessqualityof(),wherescλ0γ + ∆γ(cp )i-1(cp )i+w∆wSTPλif(cp )i-1+f(cp )iInitial Settingww’fγ(0)w (0)∆f(cp )iTest∆Controller∆λif
∆γ(cp )iManager∆γ’+wf(cp )i-1+∆wf(cp )iModel Sγ + ∆γ(cp )(State Modeli-1(cp )iof the STP)λeiscλ0
Figure1.Closedloopcontrolofthesoftware
testprocessusingreliabilitymeasurements.
istheestimateofthefailureintensityatcheckpointcp.istheexpectedfailureintensityoftheproductcomputedbyModelSatcheckpointcp.
anddenotetheactualchangesmadebythetestmanager.
TheSTPmodelandthecontrollercooperatetoformafeedbackcontrolloop.UnforeseendisturbancesintheSTPareaccountedforbyupdatingestimatesofthemodelparameters.Thecontrolloopoffersthetestmanagerop-portunitiestomakealterationstotheSTPatanycheck-point.Thesealterationscouldcomeinmanyformssuchasachangeinthenumberoftesters,inthequalityofthetestprocess,inthetestobjectivesthemselves,oranycombina-tionofthese.Thustheinherentuncertaintyandthevari-abilityoftheSTPisaccountedforinthefeedbackcontrolloop.
Theremainderofthispaperisorganizedasfollows.Sec-tion2providesanoverviewofthestatemodelthatservesasthebasisforstudyingtheimpactofdisturbancesonthecon-vergenceofthefailureintensity.Themathematicalback-groundrequiredforacompleteunderstandingofthemate-rialinthispaperisbeyondthescopeofthispaperandisfoundinstandardtextsonautomaticcontrol[4,5].Thedef-initionsofdifferenttypesofinputsignals,unforeseenper-turbationsinModelS,andtheircorrelationtotheSTPareprovidedinSection3.TheresultsofstimulatingModelSwithimpulse,pulse,step,andwhitenoiseinputsarepre-sentedinSection4andanalyzedinSection5.Section6summarizesthiswork.
2ReviewofStateModeloftheSTP1
AlinearmodeloftheSTPisbasedonthreeassumptionspresentedbelow[3].TheseassumptionsarebasedonananalogyoftheSTPwiththephysicalprocesstypifiedbya
spring-mass-dashpotsystemandthepredator-preysystem.Acompletedescriptionandjustificationofthisanalogyandthechoiceofalinearmodelisoutsidethescopeofthispa-perandisfoundelsewhere.[3]
Thewidespreaduseofdifferentialequationstomodelmanydifferenttypesofsystems[5,6]combinedwiththefactthatmostofsuchmodelsweredevelopedusinganalo-giestophysicalsystemswithassumptionssimilar[7,8]tooursfurtherjustifythechoiceofasecondorderlinearmodel.
Assumption1:Themagnitudeoftherateofdecreaseoffailureintensityofasoftwareproductisproportionaltothenetappliedeffortduringthetestphaseandinverselypro-portionaltothecomplexityoftheproduct.
.Parameter
dependsonthesoftware
projectcharacteristicsandwillhavethevalues:i-foranorganicmodeproject;ii-foransemi-detachedmodeprojectand;iii-foranembedded
modeproject.Thisclassificationandtherespectivevalues
weredefinedandempiricallyvalidatedfortheCOCOMOmodel[11].
Assumption2canbeunderstoodwithanotheranalogy.Inaspringtherestoringforceisdeterminedbythespringstiffnessandbyhowmuchthespringisextendedbeyonditsnaturallength.Increasingthespringstiffnessortheexten-sionincreasestherestoringforce.Theeffectivetesteffortcanbeinterpretedinananalogousway.Thefailureinten-sityisanalogoustothespringlength.Atthebeginningofthetestphaseislargerthanitistowardstheend.Hence,theeffectiveeffortdecreasesasdecreases.Theworkforcecanberelatedtothespringstiffness.Thelargertheworkforce,thegreatertherestoringforce,i.e.,theeffectiveef-fort.Thusspringstiffnessisanalogousto
andspringextensiontofailureintensity().InEqn.2,remainscon-stantoveraperiodandmustbecalibratedfortheprojectunderanalysis.
Assumption3:Theresistancetoadecreaseinthefailureintensityopposes,isproportionaltothethevelocityoffail-ureintensity,andinverselyproportionaltotheoverallqual-ityofthetestphase,foranappropriateconstant.
.Therefore,asmallcoefficientofviscosityis
analogoustoacarefullyconductedtestphaseandthusthenumberofnewerrorsinsertedissmall.Largercoefficientofviscosityisanalogoustothetestphaseinwhichmoreerrorsareintroducedthanwouldbeintroducedundernor-malcircumstances.Thevelocitycomponentinthedashpotisanalogoustothefailureintensityreductionvelocity().Thus,theoverallqualityofthetestphase,denotedby(),andtherateatwhichfailureintensityisdecreasing,deter-minesthefailureintensityreductionresistanceeffortwhichisanalogoustothedampingforcegeneratedbythedashpot.InEqn.3,ismerelyaconstantofproportionality.
CombiningEqs.1,2,and3inaforcebalanceequation
(
)andorganizingitinastatevariableformat()leadstothefollowingsystemofequations.
parametervalues,generatesinFigure1.UsingtherelationshipinEq.6,onecomputes.
(6)
2.1Estimationofmodelparameters
isrelativelyeasytocomputeasitisdefinedtobethenumberoftesterstestingtheproduct.Thevalueofmustbeadjustedforanypart-timeandtemporaryperson-nel.Parametersandarecomputedbyapplyingacon-vexcombination[12]ofavailablemetrics.TheremainingparametersareestimatedthroughtheuseofSystemIdenti-fication[13]techniques[14].Animportantcharacteristicofourapproachisthatestimatesofallparametersareupdatedateachcheckpointtherebyimprovingtheiraccuracywiththepassageoftime.Changesinthetestenvironment,suchasintheworkforceandthequalityoftheSTP,areaccountedforasandwhentheyoccur.[14]
2.2Computing
and
Inafeedbackcontrolsystem,thelargesteigenvalueofthesystemdeterminestheslowestrateofconvergenceanddominateshowfasttheoutputvariablesconvergetotheir
sothatitdesiredvalues.Therefore,inordertocontrol
reachesitsdesiredvaluebythedeadline,wemustadjustthelargesteigenvalueappropriately.Given,thefail-ureintensityattime,and,thedesiredfailure
later,weusethefollowingequationtointensityattime
determinetheamountbywhichtoadjustthelargesteigen-.2value
(7)
Weknowthevaluesof,andatcheck-pointcpandhencewesolveEq.7andfindthe.Theeigenvaluesofasystemaredefinedvalueof
bytherootsofthecharacteristicpolynomial(
).Computingthecharacteristicpolynomial
ofourmodelleads
(8)
whereand.UsingEq.8we
computethevariationsintheworkforceandinthequalityoftheprocessnecessarytomeetthedesiredqualityobjec-tivebythedeadline.
Fd
1/∆t
Impulse Input
∆t 0Example: Fd modeled as the replacement
of a component of the system
∆ttimeFd
Pulse Input
∆t = constant
Example: Fd modeled as the time tomigrate the system from the
∆t
timedeveloper’s to the user’s environment
Fd
Step Input
Example: Fd modeled as an increasein the communication level of thetimetest team for the remaining period
Fd
White Noise Input
Example: Fd modeled as a combination
of unforeseen perturbations thattimeoccur during the STP
Figure2.Differenttypesofinputstorepresent
unforeseenperturbationsinaSTP.
backupthatcausesthelossofimportantfilesforoneday,seemstobeareasonabledisturbancetobemodelbyanim-pulse.Supposethatatthebeginningofthedaythefail-ureintensityisandattheendoftheday,priortothepoweroutage,thefailureintensityis,for.Aninstantaneousstimulusduetothepower
outagewilldrivethesystemfromstate
to.
AnotherexampleofanimpulseinputfortheSTPisthereplacementofapre-testedcomponentofthesoft-wareproductbyadifferent,thoughsupposedlyfunction-allyequivalent,component.Suchreplacementmayoccurduetoaneedtoimprovetheperformanceoftheapplicationundertest.Asumingthatmostdefectsintheoldcompo-nenthavebeenremoved,thisreplacement,whichservesasaninstantaneousstimulus,willlikelyincreasethefailureintensityfromthecurrentleveltoahigher
level
,whereisanestimateofthefailureintensityofthenewcomponent.Onemightar-guethatthenewcomponentmaybemorereliablethantheoneitreplaced.However,weassumethatthisisnotlikelysincebothcomponentsweredevelopedundersimilarcir-cumstancesandthesecondcomponenthadtomeettheper-formancerequirements.Therefore,inthiscase,weneedto
computetheinput
inEqn.4thatwilldrivethesystemfromto.Thenextsubsec-tionaddressesthisissue.Intheremainderofthispaperweusethescenarioofthissecondexampleasthecauseofan
impulseinputsignaldisturbingtheprocess.Computingtheresponsetoanimpulseinput
ThegeneralformatofanimpulseinputispresentedinEqn.9.SincethevaluesoftheDirac-deltafunctionareknown,weneedtocomputethevectorofcoefficientsof.
(9)
whereistheDirac-deltafunctionandisthe
derivativeof.
Theorem1(CharacterizationoftheSolution)[6]:Forthe
timeinvariantsystemdynamics
andagiven,supposethattheinputassumestheformatasde-scribedinEqn.9.Letbethecontrollabilitymatrix.Then
(10)
Theorem1relatestothevaluesofand,andthecontrollabilitymatrix,butitdoesnotasserttheexistenceofthevector.Theexis-tenceofasolutionisaddressedinthefollowingcorollary.
Corollary[6]:Letbe.Foreach,thereexistsauniqueimpulseinput,definedinEqn.9,whichwilldrivethe.
toifandonlyif
TheMATLABfunctioninFigure3isusedtocompute
thevectorofcoefficients
fortheimpulseinputforthesystemdescribedinSection2.
functionret=Comp_Impulse(sc,zeta,wf,...
b,chi,gamma,x0,x1)
A(1,1)=0;A(1,2)=1;
A(2,1)=-(wf*zeta)/(scˆ(1+b));A(2,2)=-(chi)/(sc*gamma);B=[01/sc]’;D=[0];
C=[10];
sys=ss(A,B,C,D);Q=ctrb(sys);X=x1-x0;
ret=pinv(Q)*X;return
Figure3.MATLABfunctionforthecomputa-tionof
thatcomposestheimpulseinputtodrivethesystemfromstateto.
3.4Pulseinput
Thedifferencebetweenanimpulseandapulsesignalistheirrespectivedurations.Whereasisafinitenon-zeroquantityforapulse,ittendsto0foranimpulse.Thefre-quencyofapulsemightalsovary.ThoughthestudyoftheresponseofModelRtopulsesofdifferentfrequenciesisanimportantexercise,wefocusonasinglepulseoffixedlength.
Apulsemaybeusedtomodelanunexpectedone-halfdaytrainingsessionfortheentireorpartofthetestteam.Inthiscase,thedisturbanceisassumedzeropriortotime
,i.e.beforethestartofthetrainingsession.
assumesapositivenon-zerovalueat.Thisvalueofiskeptconstantovertheentiredurationofthetrainingafterwhichisresettozero.ThestructureofasinglepulseinputsignalisdepictedinFigure2.
Anotherexampleofadisturbancethatismodeledasapulseisthemigrationoftheproductfromthedeveloper’senvironmenttotheuser’senvironment.Suppose,forexam-ple,thatthemigrationperiodisoneweek.Thetestprocessdoesnotprogresswhilethesystemisundermigrationandapulseisanappropriatemodelforthisdelay.Thedifferencebetweenthisexampleandthetrainingsessionisinthesideeffectsrelatedtotheuser’senvironment.Thatis,theesti-matedfailureintensitywillmostlikelyincreasewhentheproductistestedintheuser’senvironment.
3.5Stepinput
Astepinputisapulsewithaninfinitewidth.Inourstudy
astepinputismodeledbysetting
tozeropriortosometime
andsettingittosomeconstantsoonafter.AstepinputisshowninFigure2.
OnedisturbanceinSTPthatcanbemodeledusingastepinputistheincreaseinthecommunicationanddoc-umentationlevelduetothereplacementofthetestman-ager.Assumethatthenewtestmanagerrequiresadditionalregularmeetingsanddemandsmoreeffortincollectingdataanddocumentingtheprocess.Thoughthesechangesmayincreasetheoverallqualityoftheprocess,theymayalsoslowitdown.Thereisatradeoffinhowmuchcancommunicationanddocumentationincreasewithoutslow-ingdowntheprocess.Inthiscaseweassumeanover-communication/documentationresultingindeceleration.
3.6Whitenoise
Whitenoiseisarandominputsignalthatneverrepeatsandhasaflatfrequencyspectrum.Usually,thereisalargenum-berofsourcesofdisturbancesinaSTP.Acombinationofdisturbancesgeneratedbythesesourcesismodeledaswhitenoise.Sourcesofdisturbancesthatareassumedtocombine
intoawhitenoiseincludepersonalproblemssuchasillness,software/hardwareproblemsrelatedtotheenvironmentinwhichthetestisbeingconducted,andafatalfailureofacriticalhardwarecomponent.Theseproblemsmayormaynotoccurandthereisnoeasywaytopredictthefrequencyorintensityofanyofthemortheircombination.Therefore,arandomsignal(whitenoise)seemstobeanappropriatemodelofsuchunforeseenperturbationsintheSTP.
4Results
Impulse Input100no disturbanceimpulse input disturbance at t=1090impulse input disturbance at t=50impulse input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0050100150t − timeFigure4.Effectofanimpulseinputonthefailureintensityofasoftwareproduct.Theimpulseismodeledsoastocausea5%in-creasein
atthetimeofoccurrence.TheimpulseoccursattheTADsspecifiedearlier.
Theimpactofvariousdisturbancesontheconvergence
ofthefailureintensity
wasstudiedbysettinginModelRtoappropriatevaluesandsolvingthemodelfor
.Theimpactwasstudiedinisolationforeachofthefourinputtypes,namelyimpulse,pulse,step,andwhitenoise.Eachinputwasapplied(i)earlyinthetestpro-cess,(ii)somewhereinthemiddleofthetestprocess,and(iii)closetotheendofthetestprocess.Thisvariationinthetimeoftheapplicationoftheinputallowsustodeter-minethedifferencesin
duetothedifferenttimeswhenthedisturbancesactuallyoccurduringtheSTP.Astheterms“early”,“somewhereinthemiddle”,and“closetotheend”arefuzzy,wearbitrarilysetthetimes(indays)atwhichthe
disturbanceisappliedto
.Werefertothesethreetimesas“timeoftheapplicationofadisturbance”or,sim-ply,TADs.
Unlessstatedotherwise,thefollowingparameterval-uesareassumedduringthecomputations:theworkforce
Proportinal Pulse Input100no disturbancepulse input disturbance at t=1090pulse input disturbance at t=50pulse input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0050100150t − time
Figure5.Effectofapulseinputonthefail-ureintensity.ThepulseisappliedatdifferentTADs.TheinputateachTADareequivalent
to,respectively,
.Thedurationof,eachpulseandis8
days.
,qualityofthetestprocess,complex-ityoftheapplicationundertest,modelparame-ters
,,and.Theparameterval-ueshavebeenselectedarbitrarilyandkeptconstantinthisstudy.Thedesiredreductioninthefailureintensityismea-suredintermsofpercentoftheinitialvalue,exactvaluesofthefailureintensityarenotofconcerninthisstudy.Itisassumedthatthetestmanagerisinterestedinreducingthefailureintensityto5%ofitsinitialvalue.Thus,forexample,iffailures/daythenfailureperday,wheredenotesthedeadlinebywhichthesystemtestistobecompleted.
4.1Impulseinput
Animpulseinputmodels,forexample,thereplacementofanalreadytestedcomponentbyanuntestedone.Weas-sumethatsuchareplacementcausesanincreaseof5%in
soonafterthedisturbanceoccurs.Eqn.9isaprecisemodeloftheimpulseinput.Toobtainthevalueof
inresponsetotheimpulseinputweneedtocomputethevectortodrivethesystemfromitscurrentstate
to.Comput-ingthesevaluesusingthescriptfromFigure3resultsinthe
followingimpulseinputsfor
.
Figure4showstheeffectsofvariousimpulseinputson.Asobservedfromthisfigure,theexpectedtimetoreduceto5%ofthevalueatthestartofthetestprocessis107days.However,whenanimpulseoccursattimethedesiredreductiontakesplacewithadelayof5days.Thecorrespondingdelaysinthereductionofto5%ofitsinitialvalueare,respectively,12and28days,forand.
Constant Pulse Input100no disturbancepulse input disturbance at t=1090pulse input disturbance at t=50pulse input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0020406080100120140160180t − timeFigure6.ResultsoftheperturbationofthesystemsbyanpulseinputsignalatTADs.Theinputsignalgeneratedisequivalentto
forallthethreetimeinstancesandit
persistsfor8days.
4.2Pulseinput
TwoexamplesofdisturbancesinSTPthatcanbemodeledbyapulsearepresentedinSection3.4.Nextwedetermine
theimpactofthesedisturbanceson
.AsshowninFig-ure5,atrainingperiodof8daysfortheentiretestteamde-laystheprogressofthetestprocessbythesamenumberof
days.Thevalueof
iscomputedbygeneratinganequiv-alent,thoughopposite,forcetotheeffectivetesteffort,
i.e.,
.Noticethattheabsolutevalueofisnotaconstant,butitsrelativevalueisproportionaltothevalueof.
UnlikethebehaviorexhibitedinFigure5,thevaluesofinFigure6remainconstantoveraperiodof8days.For
thethreetimeinstancesused,iscomputedastheequiva-lentof.,i.e.,
Therefore,theabsolutevalueofisthe
sameforthethreeperiodsandtherespectivedelaystoachievethesamegoalare9,21and44daysasisobservedfromFigure6.
4.3Stepinput
Anincreaseinthecommunicationoverheadanddocumen-tationismodeledasastepinput.ThestepinputisappliedtotheSTPatTADsspecifiedearlier.Twodifferentwaysareusedtotogeneratethestepinput.Thefirstwayrep-resentsanincreaseincommunicationequivalentto60%oftheeffectivetesteffort,i.e.,
andisnamed“proportional
step.”TheeffectsofthisstepinputonareshowninFigure7wheredelaysof43,33,and16daysareobserved,respectively,forthethreeTADs.
Proportional Step Input100no disturbancestep input disturbance at t=1090step input disturbance at t=50step input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0020406080100120140160180t − timeFigure7.ResultsoftheperturbationofthesystemsbyanstepinputsignalatTADs.Theinputsignalsforthesetimeinstancesare
equivalentto
,and,respectively.
Thesecondwaytogeneratethestepinputistouseanabsolutestepinput.Inthiscase,iscomputedasaforceequivalentto15%oftheeffectivetesteffortattime,i.e.,
.Theeffectsofstepinputonareshownin
Figure8.
4.4Whitenoiseinput
Wemodelwhitenoisebysetting
.Parameter
is
Constant Step Input100no disturbancestep input disturbance at t=1090step input disturbance at t=50step input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0020406080100120140160180t − timeFigure8.ResultsoftheperturbationofthesystemsbyanstepinputsignalatTADs.Theinputsignalgeneratedisequivalentto
forallthethreetimeinstancesanditpersistsfortheremainingperiod.
distributednormallyovertherange.Themeanvalueoffluctuatesaround1withastandarddeviationofap-proximately0.4foreachofthethreecases.TheeffectsofapplyingawhitenoiseatTADsareshowninFigure9.Thedelaysassociatedwitheachofthethreecasesis60,47,and24,respectively.
5Analysis
Wenowanalysetheresultspresentedabove.Ofprimaryconcernisthedelayassociatedwitheachtypeofinput.Anysideeffectsassociatedwiththeinputsignalarealsoconsid-ered.Thedelaysduetovariousdisturbancesaresumma-rizedinTable5.
Impulseinput
WeobservefromFigure4thattheimpactofanimpulse
inputthatdrives
attoincreaseswiththeTAD.Toexplainwhysupposethatcom-ponentisreplacedbycomponent.Duringtheearlyphaseofthetestcycleitislikelythatanditsinterfacewiththeothercomponentshasnotbeentested.Thus,itisreasonabletoassumethatthechangeinfailureintensity
duetothereplacementof
bywillbeonlyduetothepossiblypoorqualityofleadingtoa5%increaseinthefailure.However,laterintheprocess,additionaltestinghasoccurredandthefailureintensityoftheapplicationislikelytobemuchlessthanwhatitwasduringtheearlypartof
White Noise Input100no disturbancewhite noise input disturbance at t=1090white noise input disturbance at t=50white noise input disturbance at t=908070ytisne60tni er50uliaf 40− λ302010desired λ0020406080100120140160180t − time
Figure9.Effectsofwhitenoiseonthefail-ureintensity.,whereis
distributednormallyover
.thetestcycle.Thus,relacementatthistimecausesalargerincreaseinthefailureintensityandisduetofailuresasso-ciatedwithandwiththeinterfacesbetweenandtheremainderofthesystem.
Figure10showsthedelayintheconvergenceofthefail-ureintensitytoitsdesiredvaluewhichis5%ofitsinitialvalue.Thedelayincreaseswiththetimeatwhichtheim-pulseisapplied.Eventuallythefailureintensitydoescon-vergetoitsdesiredvalueasinFigure10(a).Replacementof
acomponentwhen
isclosetozeroleadstoanincreasein.Thetimetogetbacktoitsvalueprevailingjustbeforethisincreaseisnearlyconstantandexplainstheeven-tualconvergenceinFigure10(a).ThelaterthereplacementofacomponentthelargertheinterfacefailuresintroducedleadingtotheovershootbehaviorinFigure10(b).Similarargumentjustifiesthestabilizationofthedelay.
Pulseinput
Aspointedoutearlier,thepulseisrepresentedintwodif-ferentways.ThedelayduetoproportionalpulseinputdoesnotchangewithTAD.Toexplainwhy,supposethatsuchdisturbanceisduetoatrainingperiodof8daysfortheen-tiretestteam.Thetrainingwilllikelydelaytheprocessbythesamenumberofdaysbutwillnotincreaseordecreasethefailureintensity.Therefore,thetimewhenitoccursdoesnotimposeanysideeffectonthetestprocessascanbeno-ticedfromFigure5.Notethattrainingwilllikelyincreasethequalityofthetestprocess()andthusspeedupthecon-vergenceoftoitsdesiredvalue.However,inthisstudyweretaintoitsoriginalvalueandhencetheproportional
Table1.Delay,measuredin“days”,intheconvergenceofthefailureintensityto5%ofitsinitialvalue.Thedisturbancesoccurat
times
.Theexpectedtimeforconvergencewithoutanydisturbanceis107days.
Typeof
Delay
52899944431673736024
pulsecausesaconstantdelayregardlessofthetimeofits
occurrenceintheSTP.
Theconstantpulseinputisrelatedtothemigrationofthesystemfromthetestenvironmenttotheuser’senvironment.Inourstudyweassumedatotalof8daysforthemigration.AsseeninTable5,thedelayinconvergenceisaffectedbyTAD.Thisisduetothesideeffectsduringmigration.Dur-ingmigration,atthebeginningofthetestprocess,mostofthefeaturesrelatedtotheenvironmenthavenotbeentestedandhencethedelayinthecompletionofthetestispropor-tionaltothemigrationtime,whichis8daysinthiscase.Ifthemigrationoccursinthemiddleoftheprocess,i.e.at
,someoftheenvironmentfeatureshavealreadybeentestedandanincreaseinresultsduetothedifferencesbe-tweenthetwoenvironmentsthatexistbeforeandafterthemigration.ThisincreaseinresultsinanincreaseinthedelayasnoticedinFigure6.Anyincreaseinisconsid-eredanovershootbecauseweassumethatthemigrationpersedoesnotaffect.
AbehaviorsimilartotheonepresentedbytheimpulseinputinFigure10isobservedinFigure11.Thedelayin-creaseswithTADuntilitstabilizesatacertainlevelasde-pictedinFigure11(a).Figure11(b)isthecounterpartofFigure10(b)fortheimpulseinput.Thedifferencesarenotintheshapeofthecurve,butinthevalues.Thedelaysfortheimpulseandpulseinputsstabilizeat158and187days,respectively.Also,theovershootassociatedwiththepulsereachesamaximumvalueof15.62,almostfivetimeslargerthantheovershootof3.29fortheimpulseinput.Thisbe-haviorisconsistentwithwhatonemightexpectandisduetothelongerpersistenceofthepulsewhencomparedtothatofanimpulseinput.
(a)
200
150
yale100
d5000
50
100
150
200
250
300
350
400
impulse time(b)3.53to2.5oh2sre1.5vo10.50050100150200250300350400impulse timeFigure10.(a)-Delayassociatedwithanim-pulseinputstimulatingthesystematacer-taintime.(b)-Overshootassociatedwithanimpulseinputstimulatingthesystematacer-taintime.
Stepinput
Inourstudythestepinputmodelsanincreaseinthecom-municationoverhead.Fortheproportionalstepinput,assumesavalueproportionaltoandinflictsalargedelaywhenitoccursearlyintheprocessbecauseitpersiststhroughouttheSTP.Thedelayassociatedwithastepinput
startsat43dayseventhoughtheinputoccursat
.Thedelaydecreasesto33and16days,respectively,when
thestepinputoccursat
and.Figure12showsthedecreaseinconvergencedelayasTADmovesto-wardstheendoftheSTP.Weareawareofthetradeoffinincreasedcommunicationandhereweconsiderthelevelofcommunicationhascrossedtheborderofbeingbeneficial.Thereisnoovershootduetoastepinputasonewouldnotexpectanyincreaseinthefailureintensityduetoanincreaseincommunicationoverhead.
AsaturationeffectcanbeobservedinFigure8whenthestepinputassumesaconstantvalue.Thereisnoconditionlinkedtotheincreaseofthecommunicationlevelthatwouldleadtoasaturationinthetestprocess.Itiswellknownthatsuchbehaviorisrelatedtothecriteriausedandthequalityofthetestprocess.Itisinappropriatetomodelthecommu-nicationoverheadusingaconstantvaluestepinput.
Whitenoiseinput
Figure9showsthatthedelayassociatedwithTADofawhitenoiseinputrangesfrom0to2timesthevalueoftheeffectivetesteffort.Asexpected,theearlierthenoisestarts
(a)200150yale100d500050100150200250300350400pulse time(b)2015toohsr10evo50050100150200250300350400pulse timeFigure11.(a)-Delayassociatedwithapulseinputstimulatingthesystematacertaintime.(b)-Overshootassociatedwithapulseinputstimulatingthesystematacertaintime.
thelongerthedelay.Thedelayis60dayswhenthenoise
signalisappliedat
anddropsto47and24days,respectively,forand.However,webelievethatthedisturbancesthatcanbemodeledaswhitenoisearemoreprevalentduringtheearlypartoftheSTPthanduringitslaterpart.Therefore,thefrequencyofthenoiseseemstobeamoreinterestingparametertoconsider.
Figure13showsthedelayassociatedwithawhitenoiseinputappliedstartingalwaysatthestartoftheSTP.Thex-axisrepresentshowfrequentlythedisturbancemodeledbythewhitenoiseoccursandrangesfrom0to1,i.e.,from0%to100%.Asbefore,thevalueofthenoiseiscomputed
as
,forrangingfrom0to2.AsobservedfromFigure13,thedelayincreaseswiththefre-quencyofthewhitenoiseinput.Thisbehaviorappearstobeconsistentwithreality.
6Summary
Astudywasundertakentoexaminetheimpactofvarioustypesofdisturbancesthatmightoccurduringasystemtestphaseontheconvergenceofthefailureintensityoftheprod-uctundertest.Disturbancesaremodeledasimpulse,pulse,step,andwhitenoiseinputstotheSTP.TheseinputsarethenappliedtoastatemodeloftheSTP,proposedinanearlierwork,andtheeffectobservedonthefailureinten-sity.Resultsconfirmthecommonlyobservedphenomenonthatthedelayintheconvergenceofthefailureintensitytoitsdesiredvaluedependsonthetypeoftheinputanditstimeofoccurrence.Theuseofthestatemodelassistinthe
5045
4035
30
yale25d201510
50
0
20
40
60
80
100
120
step time
Figure12.Delayassociatedwithastepinputstimulatingthesystemattime.
quantificationofthedelays.Thestudywasconductedbyisolatingvarioustypesofdisturbances.Thisallowedustoindividuallystudytheimpactofeachtypeofinputontheconvergenceofthefailureintensity.
References
[1]J.W.Cangussu,R.A.DeCarlo,andA.P.Mathur,
“Feedbackcontrolofthesoftwaretestprocessthroughmeasurementsofsoftwarereliability,”inProoced-ingofInternationalSymposiumonSoftwareReliabilityEngineering,(HongKong),pp.232–241,November2001.
[2]M.Donnely,B.Everrett,J.Musa,andG.Wilson,
HandbookofSoftwareReliabilityEngineering,M.R.Lyu,Editor,ch.“BestCurrentPracticeofSRE”,pp.219–254.NewYork:McGraw-Hill,1996.[3]J.W.Cangussu,R.A.DeCarlo,andA.P.Mathur,
“Aformalmodelforthesoftwaretestprocess,”IEEETransactiononSoftwareEngineering,vol.28,pp.782–796,August2002.[4]G.C.Goodwin,S.F.Graebe,andM.E.Salgado.,
Controlsystemdesign.UpperSaddleRiver,NewJer-sey:PrenticeHall,2001.[5]D.G.Luenberger,IntroductiontoDynamicSystems:
Theory,modelsandapplications.JohnWiley&Sons,1979.[6]R.A.DeCarlo,Linearsystems:Astatevariableap-proachwithnumericalimplementation.UpperSaddleRiver,NewYork:Prentice-Hall,1989.
605040yale30d2010000.10.20.30.40.50.60.70.80.91white noise frequencyFigure13.Delayassociatedwiththefre-quencyawhitenoiseoccursduringthetestprocess.
[7]F.Lanchester,“Aircraftinwarfare,thedawnofthe
fourtharm,”Constable,London,1916.[8]P.A.Samuelson,“Interactionsbetweenthemultiplier
analysisandtheprincipleofacceleration,”Rev.Eco-nomicStatistics,vol.21,pp.75–78,May1939.[9]J.Musa,SoftwareReliabilityEngineering.McGraw-Hill,1999.[10]A.P.Mathur,F.D.Frate,P.Garg,andA.Pasquini,
“Onthecorrelationbetweencodecoverageandsoft-warereliability,”inProceedingsoftheSixthInterna-tionalSymposiumonSoftwareReliabilityEngineer-ing,(Toulouse,France),pp.124–132,IEEEPress,Oc-tober24-271995.[11]B.W.Boehmandet.al.,SoftwareCostEstimation
withCocomoII.PrenticeHall,2000.[12]S.R.Lay,ConvexSetsandtheirApplications.New
York:JohnWiley&SonsInc.,1982.[13]L.Ljung,Systemidentification:Theoryfortheuser.
EnglewoodCliffs,NewJersey:Prentice-Hall,1987.[14]J.W.Cangussu,R.A.DeCarlo,andA.P.Mathur,
“Astatemodelforthesoftwaretestprocesswithau-tomatedparameteridentification,”inProceedingsofthe2001IEEESystems,Man,andCyberneticsCon-ference(SMC2001),(Tucson,Arizona),pp.706–711,October2001.
因篇幅问题不能全部显示，请点此查看更多更全内容