ExAnte and ExPost Evaluations:TwoSidesof …

1 EJRR1|201547 SymposiumonPolicyEvaluationintheEUExAnte andExPostEvaluations:TwoSidesoftheSameCo in?TheCaseofGenderMainstreaminginEUResea rchPolicyLutMergaertandRachelMinto*Thisa rticleengageswithtwothemesofcontemporary EUgovernance:theroleofevalua-tionswithin aneffectiveandcoherentpolicy-makingproce ssandtheEU sconstitutionalisedcommitmenttopromoting genderequalityinallitsactivities(Article 8 TFEU).Itfocusesontheinterfacebetweenexan teandexpostevaluationandthecontributiono fevaluationstopolicylearning, , ,theEuropeanCommissionisdevelopingarenew edsetofGuidelinesforEvalua-tion, , , , ,indeed, ,thisarticleengagesanotherhorizontalthem ewithinEuropeangovernance,namelytheEU sconstitutionalisedcommitmenttopromoting genderequalityinallitsactivities(Arti-cl e8 TFEU),knownas gendermainstreaming .Giv-enitscross-sectoralapplicability,th eproceduralre-quirementsattachedtogender mainstreamingarethusrelevantinallEUpolic y, ,3thisarticlefocusesonEUresearchpolicyas anareawhichpro-videsamorepositiveexample ofgendermainstream-ingintheEU4andwhich,g iventheexpenditureat-tachedtoit, ,aroundthereiterationsoftheResearchandTe chnologicalDevelopment(RTD)FrameworkPro- grammes(FPs)(thekeyinstrumentinthisarea) ,for*LutMergaertisaResearchDirectoratYel lowWindowandholdsaPhDinManagementScience sfromRadboudUniversityNi-jmegen; ,undertheEuropeanUnion'sSeventhFramework Programme(FP/2007-2013) LASI( Law,scienceandinterestsinEuropeanpolicy- making ).

2 1 SeeEuropeanCommission, ConsultationonthedraftCommis-sionEvaluat ionPolicyGuidelines ,10 April2014,availableontheInternetat< >(lastaccessedon23 May2014). torOliveiraandPauloPinho, Bridgingthegapbetweenplan-ningevaluation andprogrammeevaluation:Thecontributionof thePPRmethodology ,17(3)Evaluation(2011), , MainstreamingGenderintheEuropeanUnion:Ge ttingtheIncentivesRight ,7(1)Com-parativeEuropeanPolitics2009, ;EmanuelaLom-bardoandPetraMeier, GenderMainstreamingintheEU:Incor-poratin gaFeministReading? 13(2)EuropeanJournalofWomen'sstudies(200 6), ;TeresaRees, Reflectionsontheunevendevelopmentofgende rmainstreaminginEurope ,7 InternationalFeministJournalofPolitics(2 005), ;MariaStratigaki, GenderMainstreamingvsPositiveAction:AnOn goingConflictinEUGenderEqualityPolicy ,12(2)EuropeanJournalofWomen'sStudies(20 05), (2009),ibid;MarkPollackandEmilieHafner-B urton, MainstreaminggenderintheEuro-peanUnion ,7(3)JournalofEuropeanPublicPolicy(2000) , ;TeresaRees, MainstreamingGenderEqualityinScienceinth eEuropeanUnion:the'ETANR eport' ,13 GenderandEducation2001, jlund, EvaluationintheEuropeanCommission-Foracc ountabilityorlearning?

3 , ,pleaseseeErikArnold,BeaMahieu,JamesStro yan,DavidCampbell,MalinCarlberg,FloraGia racca,AndrejHorvath,ZsuszaJ vorka,PaulaKnee,IngeborgMeijer,SabeenSid iqiandCarolineWagner, Un-derstandingtheLongTermImpactoftheFram eworkProgramme ,FinalReportTotheEuropeanCommissionDGRes earchbytheEuropeanPolicyEvaluationConsor tium(EPEC)2011, |201548 SymposiumonPolicyEvaluationintheEUwhicht herearemultiple, ,dif-ferenttypesofevaluationshavebeenund ertaken:exanteandexpost; ,howandtowhatextentareexanteandexposteva luationsinEUresearchpolicylinkedcoherent lyandeffectivelywithrespecttogenderequal ity?Secondly,whatrolehas(expostandexante /generalandgender-specific)evaluationpla yedintheprocessofgendermainstreaminginEU researchpolicy?Researchwaspursuedthrough atext-basedanalysisofselectedevaluationd ocuments, , , ,organisedasfollows:1)mis/matchingevalua tions;2)non-linearpolicylearning;3)diffe rencesbetweendifferentevaluationtypes;an d4) , ,whilstthereareexamplesof(selective)cohe rencebetweenevaluations,broadlyspeakingt heformativepoten-tialofevaluationsishamp eredbyprocedural, ,withre-specttogenderequalityinparticula r, ,thescholarshiponpolicyeval-uationintheE Uispredominantlyseparatedintotwocampswhi chattendtoexanteevaluation(namelytheComm ission sIntegratedImpactAssessmentsys-tem)andex postevaluation(withaparticularmethodolog icalfocus) ,intermsofatheoreticalunderpinningforthi sempiricalstudy,itwasnecessarytohuntmore broadlybeyond evaluationscholarship ,theliteratureonevidence-basedpolicy-mak ing(EBPM) , ,commonly.

4 Agendasetting,formulation,de-cision-maki ng,implementation, ,theuseofknowledgegainedthroughexposteva luationsinexanteevaluationsisacruciallin kinthepolicycycle, ,however,beencrit-icised,9inpartbecauset hetheoreticalaccountofthepolicycycle withdistinct,neatlycompartmen-talisedsta ges , ,theEBPM approachtopolicy-makingwasadoptedintheUK inthelate-1990s,withotherOrganisationfor Econom-6 StijnSmismans, PolicyevaluationintheEU:thechallengesofl inkingexanteandexpostappraisal , , ,MendeltjevanKeulenandPaulStephen-son,An alyzingtheEuropeanUnionPolicyProcess(Bas ingstoke,NewYork:PalgraveMacmillan,2011) .8 IanSanderson, Evaluation,policylearningandevidence-bas edpolicymaking ,80(1)PublicAdministration2002, stagesapproach , ,MendeltjevanKeulenandPaulStephenson(201 1),supra,note7, |201549 SymposiumonPolicyEvaluationintheEUicCo-o perationandDevelopment(OECD) ,withparticularreferencetothelinkingofex postandexanteevalu-ations,thislinearunde rstandingofthepolicyprocesscanbeseeninEU policytexts, (IAB)havemaderepeatedreferencestotheuseo fresultsfromexpostevaluationsinthe(exant e) ,intermsofeffectivegovernanceforthepromo tionofgenderequality, , ,intheory,policy-makingcanadvancetheEU scon-stitutionalisedcommitmenttogendereq uality(Arti-cle8 TFEU).

5 Evaluationscholarshave,however, ,whichmay(partly) differentevaluationscopesandcontexts,dif ferentrelationshipsbetweenresearchandpro fessionalpractice,anddifferenttimings .13 Notably, ,andshould(thus) ,giventhecomplexityandscopeoftheEU sRTDF rameworkProgrammes, : Intheevaluationofcomplexissues,judg-ment comesfrommanyindependentsources,throughm anytechnicalmeans,usingmultiplecrite-ria embeddedindifferentvaluesystems .16 Further-more, , [t] .18 Thisindicatesthat,aspartofaneffectiveeva luationprocessthatseekstocontributetogen derequality, ,basedonexistingscholarship,itcanbeassum edthatthereareclearchallengestoensuringc oherentpolicylearningoverthemultipleiter ationsoftheEU ,providingnewem-piricalresearchontheprac ticeofevaluationintheEU,aspartoftheEU :EUResearchPolicyandGenderMainstreamingF rommorehumblebeginningsintheearlydaysofE uropeancooperation,EUresearchpolicydevel opedintoawell-establishedareaofEUactivit y, , Turtlesallthewaydown:boundedrationalityi nanevidence-basedage ,33(5)PolicyStudies(2012), ,etsqq.

6 , hme, MuchAdoaboutEvidence:ReflectionsfromPoli cyMakingintheEuropeanUnion ,3(1)PlanningTheo-ryandPractice(2002), ,2010,2011, torOliveiraandPauloPinho(2011),supra, torOliveiraandPauloPinho(2011),supra,not e2, torOliveiraandPauloPinho(2011),supra,not e2;RaynaldPineault,PaulLamarche,Matie-Do miniqueBeaulieu,etal., ConceptualandMethodologicalChallengesinP roducingRe-searchSynthesesforDecision-an dPolicy-Making:AnIllustrativeCaseinPrima ryHealthcare ,16(2)Evaluation(2010), ;MurraySaunders, Theuseandusabilityofevaluationoutputs:As ocialpracticeapproach ,18(4)Evaluation(2012), (2012),ibid, , Synthesisinpolicyimpactassessment ,17(2)Evaluation(2011), , ,JuliaEspinosaFajardo,CarmenMorme-neoCor tes,Mar aBusteloRuesta, ,(SpanishMinistryofForeignAffairsandCoop eration,SecretaryofStateforInternational CooperationandforIbero-America,GeneralSe cre-taryofInternationalCooperationforDev elopmentAffairsandCooperationSecretaryof StateforInternationalCooperationandforIb ero-AmericaGeneralSecretaryofInternation alCooperationforDevelopment,2014), , ,SecondEdition(UpperSaddleRiver(NJ).)

7 PrenticeHall,1998), ,quotedinJuanAndresLigeroLasa,JuliaEspin osaFajardo,CarmenMormeneoCortes,Mar aBusteloRuesta(2014),supra,note17, |201550 SymposiumonPolicyEvaluationintheEUitshom einDGResearchandInnovation(DGRTD), ,sincethe1997 TreatyofAmsterdam, (FP1)tothecurrentseven-yearpro-gramme(Ho rizon2020), ,itwasnotuntilFP5(1998-2002)thatgenderwa sinclud-ed,soitisherethattheanalysisbegi ns:FP5(1998-2002)asthestartingpoint,into FP6(2002-2006),FP7(2007-2013),anduptoHor izon2020(2014-2020); transitionpoints andtheexpostandexanteevaluationscap-ture dbythem, , ,20definedas notrestrictingeffortstopromoteequalityto theimplementationofspecificmeasurestohel pwomen,butmobilisingallgeneralpoliciesan dmeasuresspecificallyforthepurposeofachi evingequality (emphasisinoriginal).The1997 TreatyofAmster-damthenformalisedthiscomm itment,enshriningintheTreatiestheelimina tionofinequalityandthepromotionofequalit ybetweenwomenandmenasanaim,horizontallya pplicableacrossCommunity(nowUnion)activi ties(Article3(2)EC[nowArticle8 TFEU]).

8 ,21gendermain-streaminginEUresearchpolic ytookoffwithcon-ferencesandexpertgroups, analysestostudytheproblemsatstake,thedef initionofadualobjective(toenhancewomen sparticipationandtoaddressthegenderdimen sioninresearchcontent), (FP6)wasmarkedbya strongthrust for-ward, ,FP6introducedaspecificinstrument(theGen derActionPlan,or GAP )forintegratingagen-derperspectiveinthe( biggest)fundedprojects, ,how-ever, ,anewinitiativewasdeveloped(atoolkitonge nderinEU-fundedresearch) ,stake-holdersunitedandexertedpressureon theCommis-sion, ,giventheexpenditureattachedtoresearchpo l-icy, ,EUresearchpolicyhasdevelopedconsiderabl yinbothscaleandscope(influencingthetypes ofevaluationrequired) ,Theupsanddownsofgendermainstreaminginth eEUresearchpolicy-thegendertoolkitandtra iningactivitiesinFP7(Antwerp,Belgium:Pol icyResearchCentreonEqualOpportunities,20 11).20 CommissionCommunicationonIncorporatingeq ualopportuni-tiesforwomenandmenintoallCo mmunitypoliciesandactivi-ties,COM(96)67f inal, sCommunicationongendermainstreamingwaspu blishedin1996,whenFP4wasrunning(1994-199 8).

9 22 EuropeanCommission,Stocktaking10yearsof" WomeninScience"policybytheEuropeanCommis sion(1999-2009),(Luxembourg:Publications OfficeoftheEuropeanUnion,2010),NBThisisa lsotext#9;MergaertandDemuynck(2011),supr a, (2010),ibid, # , (Doctoraldissertation,RadboudUniversitei tNijmegen,Nijmegen,2012).25 StevenH jlund(2014),supra, , EvaluationintheEuropeanUnion:AddressingC omplexityandAmbiguity ,inJan-EricFurubo, (eds.),InternationalAtlasofEvaluation(Ne wBrunswickandLondon:TransactionPublisher s,2002), ,seeEricArnold,BeaMahieu,JamesStroyan, ,FinalReportTotheEuropeanCommissionDGRes earch(EuropeanPolicyEvaluationConsortium (EPEC),2011), |201551 SymposiumonPolicyEvaluationintheEUevolut ionpresentsparticularchallengesandoppor- tunitiesastheperiodofanalysis(1998-2014) ,tentextswereidentifiedaskeyevaluationsa ttachedtothedevelopmentofEUresearchpolic y(foralistoftheselectedtexts,pleaseseeth eannex).Thiscor-pusforanalysiscomprisedb othexanteandexpostevaluations, (IIA)accompa-nyingthetransitionsfromFP6t oFP7, ,27becoming fullyoperational in2004 ,therewasnoformalsystemofexanteassess-me ntperse, , (includingforgenderassess-ment),29asanin strumentoftheEU (FYA).

10 Theonecoveringtheperiod1995-1999(publish edin2000)wasavailablefortheshiftfromFP5t oFP6andtheother,covering1999-2003(publis hedin2004),fallswithintheboundariesofthe shiftfromFP6toFP7, (thesetookoverfromtheFYAfromFP6).Theycon sistedofanexternal,expostevaluationofFP6 (publishedin2009)andanexternal,interimev alua-tionofFP7(publishedin2010).Inadditi ontothesesixgeneralevaluationsoftheFPs,f ourgender-specificevaluation-relatedexer cis-eswereidentifiedforanalysis, (retrospective) , , ,releasedinMay2009,wastheGen-derMonitori ngStudiesreport, ,in2010,theCom-missionreleasedastocktaki ngtextregarding10yearsof WomeninScience policy(1999-2009).Thetenselectedtextsdon otproduceawhollyor- ,duetotheirdatesofrelease, ,theevaluationnarrativethatemergesdoesin -cludebothexante(two)andexpost(eight)eva lua-tions,comprisinggeneral(six)andgende r-specific(four) :1)mis/matchingevaluations;2)policylearn ing;3)evaluationtypes;and4) ,duetotheirdatesofre-lease, ,theGenderEqualityReportofFP6(text#5),th egeneralexpostevaluationofFP6(text#6)and theGenderEqualityMonitoringinFP627 CommissionCommunicationonImpactAssessmen t,COM(2002) , Evidence-basedpolicy-makinginEurope:anev aluationofEuropeanCommissioninte-gratedi mpactassessments ,24(1)ImpactAssessmentandProjectAppraisa l(2006), , ,supra,note27, : ToensurethatEUactioniseffective,theCommi ssionassessestheimpactofpolicies,legisla tion,tradeagreementsandothermeasuresatev erystage-fromplanningtoimplementationand review.