public procurement

#32 - Pedro Telles (Swansea University)

What do low value contracts, framework agreements and Brexit have in common?

Interview with Pedro Telles about low value contracts, framework agreements and what impact Brexit may have on public procurement in the near future. Pedro is a Senior Lecturer in Law at Swansea University and was previously a Lecturer in Law and Public Procurement Law Specialist at Bangor University's Institute for Competition and Procurement Studies. This is the final episode of the PPP (and a double length one) with the interview conducted by Albert Sanchez-Graells from the University of Bristol.


Thank you very much Albert for accepting the request to be the host for the podcast because it’s the last one and I wanted to finish in a slightly different note than usual, so I thought it would be interesting to actually put myself at the end of the microphone and be on the hot seat talking about procurement instead of making the questions, and putting the head on the other people.


Yes, I’m sure that you feel very uncomfortable now in that chair?

Actually, I do [both laugh].


So Pedro, I think you have lots of things about low value contracts, this is something you’ve been writing about for the last what, four or five years, so what are your thoughts on that now, how do you see it coming in the future?

Yeah, that’s true, I’ve been thinking about low value contracts, i.e., contracts which are valid below the new financial thresholds for the last five years I think, and over the years I became very surprised with the way they were treated and actually their importance in the overall picture of public procurement regulation. I would say that firstly it’s important to talk about them today in 2017 because the thresholds that now determine the size of the internal market for purposes of public procurement are going to be revised by 2019, or at least it’s expected that the commission will look into them in 2019. And it can go either way, they can go up or they can come down. They have been very stable for the last 20 years, mostly changing just because of exchange rate fluctuations between the euro and the special drawing rights of the IMF, but come 2019 we may have a more significant change in terms of what are the contracts that are covered by EU rules via the thresholds. Now the current system is perfectly arbitrary, it’s set without any specific reasons, so that the threshold values that we know and have been using for the last 20 years or so, there’s no actual substantive reason for them to have the value that they have. And in consequence is that above the thresholds you apply the full might of EU rules and below the threshold you apply essentially national rules with exception of the contracts which are considered to have the certain cross bordered interest, which I’ll mention in a second. Now one of the interesting consequences of this arbitrary distinction is that in reality only around 17 or 18% of public procurement expenditure is actually covered by EU rules. So only the tip of the iceberg is actually subject to directive 2014 24, and the other directives as well. Which has led me to question a little bit the logic of the system, which is why would we focus the most important rules in just a small subset of contracts? As time has went on and I looked into a little bit more detail with what’s happening with the contracts below the thresholds, I was even more surprised with the way that they’re treated, and going back to the topic about the certain cross border interest test that was created by the Court of Justice first, with the Telaustria case may be 15 or 16 years ago, and has been upheld with variations ever since, effectively this means that for the contracts below the thresholds they will only be subject to the EU principles, if the contract itself has or is able to attract an interest from an economic operator based in another member state.

The Court has been fluctuating a lot between what is certain cross border interest, what is not certain cross border interest, and effectively this means to me that the Court has probably painted itself into a corner. In the sense that they are enabled to clearly provide a set of rules, or guiding principles that can be consistently applicable to any given situation. And the reason for that being that by subjecting the application of the rules to whomever comes to the contract or is actually interested in contract, we are subjecting the rules of the contract, or the procedure to a condition that is uncertain at the beginning. And that for me it’s a critical mistake and it’s a critical error from the legal system because you’re not providing any sort of legal certainty, for the public procurer they start the procedure, they don’t know at that moment in time if they’re going to be covered or subject to your rules in this case, the principles from the treaty, so equal treatment, non-discrimination, transparency, or depending on who actually comes up to the contract, if they’re going to be subject only to the national rules, whatever they may be.


But Pedro, if I can push you on that, because it’s very clear conceptually, but then at the same time every system has some sort of requirement for transparency and competition and proportionality, so what is in the principles that makes people so happy about them?

I suspect that it’s probably the fact that it means that in one way or another there’ll be subject to EU law and they would prefer to be subject only to the national law, whatever that may be, I know that in many member states there is a fully fledged legal regulation for those contracts, whereas for example here in the UK there isn’t, I mean if you look at the Public Contracts Regulations, as we did with our commentary, there is some regulation but it is very light touch and very modest in a sense. So I suspect that the problem people have is exactly being subject to those principles that are coming from EU law and not from national law. And obviously applying principles, it’s always prone to a significant degree of uncertainty in terms of what actually is included and contained within that principle. Whereas if you have national rules which may contain the same or similar principles, the national rules would be… one would hope, a little bit more detailed and prescriptive. So I think there’s a degree of legal uncertainty, but the biggest criticism that I have for the role of the Court of Justice in all this process is that it has effectively created the perfect "Schrodinger’s cat" of public procurement in a sense, that any given contract below thresholds may or may not at any given moment in time be subject to EU rules, i.e., the principles, or subject only to national rules, depending on the actual economic operators that turn up to the contract, or to the procedure.


Yeah, or even that could have turn up.

Or even that could, exactly, so it’s even more complex than that. So I’m really not happy with the solution that the Court of Justice has reached and I think that it is time that the Court actually looked at it with a fresh pair of eyes and reconsidered its approach. Now looking also at where the thresholds came from originally, I wrote a paper about that a couple of years ago and I was very surprised by again not finding any particular justification for the thresholds, and if I may use an analogy on this, let’s imagine that what we take now for granted, at least for now, free movement of people, or free movement of goods or services, they apply to all situations, without any question, without any reservation, and they’ve been in the treaty since 1957. What I think is the original scene in procurement is that we don’t have a similar approach or the treaty’s do not take a similar approach in 1950s and it was only in the ‘60s via the original directives that procurement was subject to positive rules and not only negative rules. Now when those positive rules came in they came in and allowed the member states to try and negotiate a degree of protectionism, for the national markets, and we can see that in the ‘70s in the way that the original thresholds were introduced, and you can clearly see that in the ‘80s when the threshold for works which was one million ECUs at the time I think, was increased to five million on the run-up to the negotiations for the Tokyo round, or the Tokyo agreement of the GPA, where works were the included as part of the GPA, and were included precisely at that level.

So as we’d accepted from the beginning that all public contracts would be subject to free movement and subject to this possibility of any economic operator participating without any discrimination, without any question, I think that the procurement landscape would be very different and we would not have thresholds at all. Now the other level of my criticism regarding the thresholds is that at least what they are today, we effectively we’ve got an internal market that is determined by external commitments, i.e., by the commitments that the union has accepted within the GPA. So the thresholds that we have today, that they’re in the directive from 2014 and they were updated afterwards, they are the exact same thresholds that the union has accepted within the GPA. Now the GPA as an agreement, or has a group of bilateral agreements effectively, implies that we are determining the size of our internal market, not by internal pressure or by internal drivers of the union, but by our ability to negotiate with our external trading partners. And that for me doesn’t make a lot of sense, effectively what we’re saying is that the internal markets, it’s not just us inside the union that decide what is internal market, it’s our ability to compromise with our trading partners that is going to determine the internal market. Again I don’t think that makes a lot of sense, I think that the union should be more ambitious and should accept that lower value contracts today also are part of the internal market and one of the reasons I think about this is if you look at our improvements in productivity have crept up over the last two or three decades, they have made it possible to actually… economic operators in other member states to compete for more and more contracts with the lower value, and I suspect that one of the areas where this is going to be more visible quite quickly, it’s going to be with contracts that have a digital nature, they are digital by default and digital only. So services, contracts that can be delivered over the internet, whatever they may be, whatever the actual object, for me, from my perspective today they are cross border, or they are bound to generate certain cross border interests for sure, because the opportunity and transaction costs of actually bidding for those contracts and delivering those contracts is much lower than it was to sell physical goods 20 or 30 or 40 years ago. So I think that if I could accept in the ‘70s that the member states would use the argument of opportunity costs, or transaction costs to block the application of EU rules to traditional goods and services and also works, today I think it’s less likely that that argument holds, but certainly it does not hold for digital contracts.


I think it’s very interesting, I think in terms of the relationship between the GPA and the directives, and particularly here in the UK this is something that people are starting to look at in lots of detail, because of Brexit and we may talk about this later, but how do you see the convergence, because if we push EU rules to extend corporates down, would it be all the way down to any value, or would we end up with a triple layer regulation where we would be looking at the really high value contracts, we will be GPA plus full EU rules, and then a sort of weaker EU regime for lower contracts, and then still as more domestic threshold, how would you see the final map once rethinking takes place?

Well as you’ve mentioned a few minutes ago that already happens to a certain extent with many member states, so Portugal, Spain, Czech Republic, Slovakia, Finland, Poland, those are the ones I know, they already regulate contracts below thresholds, and there is I suspect in most cases a very light touch regulation for really small value contracts and that varies from country-to-country, but the vast majority let’s say from 10,000 euros or 15,000 euros up to the value of the thresholds, they’re subject to rules that are similar to the ones that are already contained in the directives, albeit slightly, one would hope at least slightly lighter. So that multiple layering of regimes already exists, what this would do if you lower the thresholds for let’s say 10% of what they are now, is that we would have a lot more harmonisation of procurement rules across Europe in terms of the coverage of contracts and coverage of procurement spends, that is out there. So whatever option we prefer, we always going to end up with multiple layers of regulation, so it’s just a question of defining where the boundary for each one of those is going to be?


Yeah, I think that’s right, I mean there is also an element of interaction that probably will bring us to the next topic, but also the more aggregation of demand there is, and the more that use is made of framework agreements and dynamic purchasing systems, then the more span of reallocated towards larger contractual mechanisms that then would be covered by the euros as well, so I think it’s right to say if you want to have a sort of homogenous spread of spend covered by EU rules, we need to move the threshold lower so that those member states that are behind in adopting these frameworks and [?? 14.19] still get more spend covered by EU rules. Otherwise finding of the ones that are more advanced in the procurement are the ones that are also more subjected to EU rules, than the other way around.

Yes, no I think you’re absolutely right, and it’s another point that is interesting to mention here in terms of currency fluctuations and in terms of value, which is the exchange rates that then determine which are the thresholds for each member state, that does not have the euro as an official currency, or is set every three years as it is for the euro denominated currencies, or the euro denominated countries, but it doesn’t take into account fluctuations doing those two years, so you may have for example a contract which in euros was covered, and I’m thinking about the UK of course, a contract that two years ago would be covered both in the Eurozone and also in the UK, that because of sharp fluctuations in the value of sterling it should no longer be covered because it’s much lower, or it’s much higher than it needs to be. So that is another problem of using money in value as a proxy, for a potential interest of a contract for an internal market, so I suspect that going forwards, I think you’re right, that if we don’t change the threshold values we’re going to end up with a system whereby you’ve got some contracting authorities, in some member states which would have most of their procurement expenditure covered with the directives, and most contract authorities in most member states, not… and that’s what’s already happening to a certain extent because as I’ve said only 17% or so of the procurement spend is actually covered by EU rules.


So yeah, I think you have plenty to write about this in the next couple of years as well.

I do, I do, I haven’t finished it off yet [laughs].


But then at the same time you’re starting a project, right, because I think you’re seeking funding for work on frameworks, so that’s the other area of interest that you have at the moment?

Yeah, yeah.


So what are your thoughts about frameworks, dynamic purchasing systems and all these crazy things?

The first thing that comes to mind when I look at framework agreements is that they are a product of their time, a little bit like the thresholds, they are a product of the ‘90s and catalogue purchasing as it was designed in the ‘80s and the ‘90s. So it’s now 2017, and when we moved from a buying process that is very time consuming because we’re buying each unit or each time you’re launching a complete new procedure, to moving to an aggregate, a system of aggregated demand like the one for framework agreements, we’re moving into effectively the paradigm of the ‘80s and the ‘90s which was what framework agreements are, whereby you have a catalogue and certain participants are selected to be part of that catalogue and supply goods and services to the contracting authorities. Now the more I look into this, the more I’m worried about the implications, so on the one hand there’s no denial that it allows for savings to be found and it allows for the reduction of the transaction costs over certain given period of time, I agree with that, I don’t have a problem with that. My questions about framework agreements is, okay, has anyone actually looked at the possibility that they foreclose the markets, because especially in those framework agreements that are mandatory for certain contracting authorities to use, then only the suppliers that actually make it in are the ones that are going to be able to supply a certain demand for any given period of time, and in fact in most cases contracting authorities prefer to have lone framework agreements.

Then if you actually look at the way the framework agreements operate, I don’t think that we can actually say that the contracts themselves are subject to EU law, we can say that the selection for the framework, yes, but from then onwards it’s pretty much anything goes, because framework agreements tend to be a black box, no-one pretty much knows what’s happening inside them, how the money’s being spent, how the contracts are being selected, it’s just as if it’s not relevant anymore. So we move from a system where every single contract is important enough as long as it has certain value, to be covered by EU rules and national rules, to a system that we’re going to select a few participants, and then contracting authority does pretty much whatever it wants.


But I think that sceptics would say, well but then once they are inside the game, then the participants are going to monitor each other and the contracting authority, so provided you have some sort of private remedy that they can exercise, to challenge the mini competitions of the calls within the framework, then we’re fine, why would that not be sufficient, what would be your reaction to that line of argument?

My reaction would be I think two-fold. The first one is, well if you don’t know what’s happening inside you don’t know how the money’s being spent. So even if you’re inside the framework, if all the money is being steered towards one of your competitors for whatever reason, you don’t know if that is happening, you don’t know necessarily if that is happening. So there’s not necessarily any accountability inside a framework agreement. So that is the first problem, then because effectively contracting authorities at this moment in time have free reign to design the framework agreements, how they operate on a day-to-day basis as they see fit, it’s very hard for an economic operator that is inside to actually complain. On certain jurisdictions, for example the UK, complaints by suppliers are perceived to be a black mark against the contracting authority and suppliers are very careful in the way that they express their complaints, so if you’re already inside a framework agreement, what incentives would you actually have to blow the whistle and where would you actually blow the whistle, I mean if you consider that framework agreements are purely private contracts, are not subject to public procurement rules at least, it becomes very uncertain how would you go about and try to challenge any decision, because technically the period where the framework agreement was subject to EU law or to procurement rules has already passed. And that’s another thing it doesn’t make much sense to me, because with the traditional procurement procedure, you look at the ones that are covered by EU rules, EU rules apply from the beginning to the end of the procedure until the contract is awarded. With framework agreements it would seem that the application of the rules pretty much ends once the framework is established, and it’s not clearly extended all the way down to the contracts being awarded. So there’s a lot of uncertainty in terms of, again uncertainty in terms of what is the legal framework that to be applicable to those contracts.

Then as you said, it is possible for the competitors that are inside the framework sometimes to know what’s happening, and that leads to another problem which is an issue of competition law, because it facilitates or it creates the condition to facilitate collusion between the participants inside a framework agreement. So at the same time whereas as I’m in favour of transparency, usually inside framework agreements I am very very aware of the risks of too much information being disclosed within the actual framework agreement, which would help bidders to coordinate their practices. And then on the long run, there is something that has not been explored well in terms of framework agreements, which is they may lead to reduced supplier diversity in the long run. Why do I say this? Because if only a limited number of suppliers makes it to the framework agreement, by definition they’re going to be the best ones in the sense of their financial, of their experience, their financial figures, so turnover, the experience, all those selection requirements, and obviously the price of not being in the framework is so high that they probably are going to try to go as low as possible, even at the dumping level, to make sure that they get themselves into the framework agreement. So by definition this makes the life very difficult for small suppliers and younger suppliers in a market to make it to a framework agreement. While it is possible to design framework agreements that are more open than this, for the most part the ones I’ve seen are very much designed to reduce competition, probably too much, and that may lead in my view as more money gets spent for them, that may lead to reduced supplier diversity on the long run. So those would be probably the biggest issues I have with framework agreements. Now most of them actually do not apply to dynamic purchasing systems, with the dynamic purchasing system you can join and leave the system at any given time, as a supplier, it’s probably a lot more transparent because it’s always open so you always know more or less what’s happening, in terms of design it can be probably made as resource intensive or not as the framework agreement.

However, if you look at the number of framework agreements, versus the number of dynamic purchasing systems, there’s no doubt whatsoever that framework agreements are much more popular today than dynamic purchasing systems, but why is that, why is the market sticking with framework agreements, it’s because they are older, they’ve been used more often, people are more comfortable with them in terms of contracting authorities, what advantages do they see, to prefer to use framework agreements over dynamic purchasing systems.


There is a movement I think in the UK, to try to transition from the framework agreements to the dynamic purchasing systems. And it was never clear to me why dynamic purchasing systems didn’t work that first time around, and I think it, the main argument is oh we had to run an open procedures, so we didn’t want to do that, whereas now that we have a negotiated procedure, restricted procedure, sorry, we can organise dynamic purchasing systems more easily, but I don’t understand that because it’s open-ended by nature, so it will always be open forever. So how do you see that?

Well maybe an issue of misconception, I mean especially here in UK we have to understand that there’s been over many years a distrust on the open procedure, and for a long time actually the restricted procedure was more popular than the open procedure and the UK was the only member state where this was happening. And for me the explanation is that the restricted procedure allows the contracting authority to reduce the number of bids that they’re going to be looking at, and allows it to introduce only the best bidders actually make it to the bidding stage, so it gives them control, for whatever reason. I mean in other countries that doesn’t happen, but here it’s clearly an issue in my view of control and control over the market. Now probably people did not understand that or they did understand that the framework agreement allows them to reach the same conclusion because only a set number of participants is going to usually make it to the framework agreement so effectively you can use the framework agreement as you would use the restricted procedure, so it’s an evolution and that sense of the procedure. Whereas the dynamic purchasing system, because it requires the open procedure, even though even with restricted procedure it’s open as you said, for everything, it’s open-ended, so participants would have to be checked as time goes on.

I suspect people just thought that well it means that we’ll have to deal with the number of bids, a number of tenders and we don’t want that, so we want to manage our workload and we want to make sure that only the best end up. As time goes on I hope this changes and I hope that practitioners and public procurement officials, not only in the UK but also elsewhere, understand that perhaps that they will be better served with something that is open ended, like the dynamic purchasing system, instead of the more closed nature of the framework agreement. I simply don’t know if there is enough take-up, bearing in mind that dynamic purchasing systems is what 15 years old already, I don’t know if there’s already enough take-up to actually reach a critical mass, that would allow public procurement officials to feel comfortable with it and start using it more often, what’s your view?


I think it’s an interesting point, I think that the issue will come up every time a framework expires and the contracting authority needs to decide whether to tender a new framework, or migrate to a dynamic purchasing system and I think you’re right, it’s a cultural issue and I think that the, probably the policy design should be maybe nudging authorities to go for a dynamic purchasing systems as a default, unless they can find some very good reason to stick to a framework agreement, and my personal view is that competition authorities should be doing a lot more in this area, to a public sector, so not only in terms of cartels which you have stressed very rightly, but also trying to educate the public sector in becoming more open in the way they dynamically purchase from the market. But I guess this is going to be a discussion again for a long time, but I think any empirical work in this area is going to be much much deserved, so I hope you do move forward with some of your current proposals.

Well that’s certainly what I’m trying to do, I’m trying to actually look into framework agreements and dynamic purchasing systems in multiple member states, and think about why are they not being used more, the dynamic purchasing systems on the one hand, and why do some member states actually use framework agreements so much, I mean UK is one example, Denmark is another example, and then you have other countries that don’t use them almost at all, and why is that, why is there this huge discrepancy in terms of practice, and I think that the cultural issue is going to be one that is going to be that the forefront, but also even thinking about framework agreements, I think that it will be possible to design framework agreements that are less damaging to competing interests, or competing principles in public procurement, so for example if we reduce the length of the framework, if we increase the number of participants, those are two things that can be easily done, if the contracting authority is willing to try them, and they will at least minimise parts of the downsides of the framework agreements in terms of the impact that they may have on the market, and especially with the SMEs. So if you know that these framework agreement will last only for one year, well if you did not make it you may have an opportunity to make it in the following year. The problem with I see, as I see framework agreements in the UK, now is that they tend to last for four years and the underlying contracts may last for longer. So there’s another problem that we haven’t discussed with framework agreements, is that while the framework agreement itself may last for four years, the underlying contracts may keep on going.


Yeah, that’s the beauty of the lack of definition of active…



You can keep on creating holes everywhere.

So I think this… framework agreements, and this whole area of centralisation of procurement as more and more money is spent via this demand aggregation tools, I suspect that we need to pay a lot of attention to them and we need to be very much on the ball in terms of looking out, they’re being used in practice and what are the trade-offs, because all these procedures and tools imply trade-offs, and I don’t think that the market and certainly the European Commission as key policymakers in this, I don’t think they’re really aware of the trade-offs that are being implied by the framework agreements over dynamic purchasing systems.


Yeah, I would agree, and I think you have plenty of work to do, but I think we have been speaking for this while about your old interests, about value of contracts and thresholds, then your new developing interests about framework agreements and dynamic purchasing systems, and then I guess we need to talk about things that don’t necessarily, are of your interest, but are on your table, everybody that does procurement in the UK, Brexit is not something we want to deal with, but hey we have to.

Yeah, we have to.


I don’t know, I mean we have spoken some times about Brexit but I don’t think we have thought together about how things are going to change or not change and why should they change, so I’m very curious to hear about what you have to say about Brexit and procurement generally, about how you think the UK is moving forward?

Right, Brexit and procurement, so I think we wrote about this, what, six months ago on our blogs, there’s a couple of entries about this.


Yeah, it feels like hundreds of years ago because this is…

Yeah, we haven’t revisited that recently. So Brexit and public procurement, the first point is why should anything change in terms of rules, there’s so much knowledge based on the current rules, there’s so many systems that have been built around the current rules, why should we change them, that’s the first question. Related with the first question is, if you look at where the government, all the governments in the past could have regulated procurement, usually the governments have been very loathed to actually use those powers of regulation, so for the first time in 2015 we’ve got some regulation for contracts building thresholds, but that is very light touch, it’s very detached in a sense, so if that is any indication it would mean that the government is probably enabled in my view to come up with a set of rules that is much better than the current ones. If you think about it, people don’t know this but we know the story or the issue behind this, many of the changes introduced in the current directive were actually pushed very hard by the British government, this may come as a surprise but there is a lot in the directive that came first and foremost from UK, obviously with the support from other member states, but the UK was leading the charge to introduce those changes. So it’s not as if all that is on the directives and in consequence, in the public contracts regulations, both the ones that you commented on and the utilities, it’s not as if there’s much there that is purely continental in a sense, it’s actually there’s a lot that comes from the UK. So those are the first two questions I would ask. So why should the branding change when so many of the rules are actually designed or influenced by the UK, and in the areas that the government could have regulated and can regulate it has not actually exercises that right. So those are the first two questions. So then we move onto the real complicated matters, which is whatever the government wants to do, it will depend first and foremost on what trade agreement it ends up having with the European Union. So if the trade agreement involves procurement, then I suspect there’s not much scope for the rules to change because if you look at Norway and if you look at Switzerland, that is the situation with them as well, so I suspect not much will change in that area. What may change is for example, and it depends where the country heads in terms of actual political direction, but we may have, as we have in Scotland and Wales today, a more clear steer for, or a more clear steer based on a certain industrial policy. We’ve been that to a certain extent with the attempts of the government to force procurers to buy British steel, I mean even in the Cameron government and now with Theresa May so we may see more of that happening, even though, and respectively of the agreement that the country ends up having with EU. And then finally the final layer of uncertainty, or legal uncertainty that may shape, or it’s going to shape how procurement is regulated after departure from the union, is the GPA commitments, is the UK part of the GPA, well at this moment in time it’s not, will it be able to inherit the schedules of the EU, maybe, will it have to negotiate with some session, probably. And if it does does it have an interest in actually committing to anything that is different from what it has now, even though the current system was negotiated by the EU on its behalf. So that will shape as well how the UK procurement framework is going to look in the future.


I think it’s interesting because there’s all these different dimensions in which things may change, so the industrial policy I think is the one that probably is going to put more pressure on the system, and then the negotiations of the GPA level, but I think that the biggest issue, and it keeps playing with my head, is why should anything change really, I mean I know what to be… what I think, and there’s something we actually have discussed, but the transposition was so minimal, that we actually lost all of the opportunities to reform the system…



…just two years ago. And I am still not clear as to whether it was in anticipation of a major reform down the line, or just because there are no clear ideas of what to do with the system that if you look at it, actually right now it’s pretty flexible, I mean the negotiation is pretty straightforward, you can introduce lots of sustainability and other requirements through your technical requirements and other things, do you think this has been sort of, everything has been held and waiting for big change to happen after Brexit, or are we just thinking about a change that may actually not get there?

I don’t think the government was holding up any changes from the 2015 regulations, I mean they spent a lot of political capital in getting the directive to be what it is, so obviously I mean bearing in mind as well, in the way that the government sets out transposition should occur, it was always going to be a very minimal transposition, but it didn’t have to be this minimal. The government opted to be very Spartan in the way that it transposed the directive into the national law, and I suspect that in addition to the stated policy, I suspect that one of the main reasons was that it was the only way they could get it out of the door before the elections in 2015. If you think about it is, it is pretty much the only reason one can conceive for the transposition of the main directive to be rushed and to be available and made available in February 2015, whereas for example the utilities one came much later, and the transposition had so many typos and errors that we already had amendments in 2016 to correct all those typos. So I don’t think that there is any actual desire to introduce a debrief form of the procurement rules. Over the years, and certainly I mean I have been in UK for ten years now, dealing with procurement, and something that has always surprised me is when anything goes wrong with any given procurement, people blame the rules and people blame the rules and in consequence they are blaming the European Union and saying, oh no, we cannot do this because of the god damn directives, or we have to do it this way because of the directives, and I was lawyer, I was a practitioner, it’s not that simple. There was already a lot of flexibility in Directives 2004 18, and there is certainly a lot of flexibility on this one, so if you’re not exercising that flexibility it’s not because you can’t, it’s because you’re not able to, for whatever reason. And you’re right when you say that for example it’s so much easier today to use negotiated forms of procedures than it was just two years ago, so what else do you want to do, I mean the only thing that I can come up with is not that was said in contracts at all, that will be probably the preferred option of…


Yeah, but that’s wrong in the UK [laughs].

Yeah, I know, I know, but that’s why we have got the rules, right, to avoid that. But I suspect that would be the preferred option for public procurement officials, which is I want to hire who I want to hire, whatever the way I want to hire.


On that note [laughs], how do you think it’s going to unfold now with this Brexit commitment requirements we discussed in our blogs yesterday, or two days ago, do you think that in this time in-between now and Brexit, they will keep on testing the waters, to see what they can do, or do you think that some point things need to stabilise or…?

For future reference, I mean these are two blog posts that we’ve written about, a cultural fit clause in a contract to be tendered by the Department of International Trade, which read something along the lines that we will want the contractor to choose to be committed to make a success out of Brexit. Now there’s two levels of that discussion Albert, I think the first level is in terms of the actual application of the existing rules in a day-to-day basis, on the one hand I hope it doesn’t change much, because if it changes it means that people are being very strategic in the way they approach procurement, that is effectively a possibility, that may happen. And the more strategic people are the more likely it is that they will just ignore the rules or at least the bits that clearly come from EU law, and the reason I say this is because over the last few months I became more and more convinced that as the UK works towards Brexit, the effect of the Court of Justice, of any action before the Court of Justice, or most remedies available, it’s going to become lower and lower, so it’s going to become weaker and weaker and people are going to start to become more embolden in the terms of, okay, if my colleagues over there have been able to tender this contract with these kind of clauses which are for whatever reason they’re illegal, why can’t I do the same, why can’t I push the boundary a little bit further. So I suspect that we’re going to see that happening unfortunately going forward. And to such an extent, if you think about before the transposition of Directive 2014 24, how Scotland and Wales were looking at community benefits, I would argue that many of those ideas would not fit within the rules as they were conceived in the 2004 directive. Do they fit today in the 2014 one, maybe, I mean we can discuss those, but I suspect that the most entrepreneur minds in a sense will continue to push the boundaries and try to see what they can get away with, because the risk of being caught is getting lower by the day, and even if they’re caught they can always play for time, and say okay, so if you want to challenge this, this decision for being illegal, go for it and take it all the way to the Court of Justice and by the time it reaches the Court of Justice we’re going to be out of the EU, so it’s not going to have a jurisdiction over the topic. That’s definitely a risk, I hope I’m wrong, but I think I’m going to see that more and more happening, not only in procurement unfortunately, but also in other areas as well. And as someone who is benefiting from free movement, of people at this moment in time, that makes me a little bit uneasy.


Yeah, I think you’re right, and I completely share your uneasiness. I think the last issue that maybe is interesting picking your brains on before we conclude is, you have been interviewing loads of early career researchers, people doing interesting research in this area for the last couple of years, and one thing you keep moaning about is methodology and how [both laugh] legal methods are not as straight as they could be and what do you want to say about this, get it off your chest.

Yeah, okay, so I’m going to make two comments on that, so the first one is it was great that I was able to interview so many people from so many different walks of life that are super smart, and are doing amazing research, related to procurement. I mean when I started this I could not anticipate the breadth of research that was being done by early career researchers on procurement, so it was great to see all that being done. So I’m incredibly grateful to the British Academy for actually giving me that opportunity, and obviously creating the obligation of me on delivering on that commitment. And the consequence of talking with so many people that are doing interesting stuff in procurement, from different sciences, different areas, different backgrounds, is I came to the conclusion of something that have been on the back of my mind for a long time, which is for the most part in law in terms of research methods we’re stuck in the dark ages, my wife she is, she’s a social scientist and she works in a medical school so she deals a lot with qualitative research methods, and we have intense arguments and discussions about research and research methods and I’m always in the losing side of it, I mean because I can’t really defend what effectively in a sense in law are essentially logical detections to such an extent, but a lot of inferred opinions and legal opinions from people that are looking at law and giving their opinion.

when I look at legal interpretations we usually evaluate the, for being right or wrong, depending on how the Courts end up applying them, which means that if you’re more persuasive than your opponent and people have a different view, you are more likely to actually be right. But that doesn’t mean that we are actually correct, and there’s an absurd I would say lack of good quality empirical research in law. Now over the years, there’s been an increase, and it’s important to mention that, an increase in the use of social sciences research methods to look into legal questions, and I use those for example in my PhD so I was looking at the law in action, how it was being actually applied in real life and what were the problems that people were having in real life. So it was a step forward in comparison with just, you know, traditional argumentation or traditional black letter analysis. But what I think is missing in law, and certainly in Europe, it’s happening a little bit in US, but it’s happening in Europe, is an ability and an interest and a commitment to do quantitative research studies in law, this is different from let’s say law and economics, because mixing law and economics means that you may think both research methods and research questions, so you’re looking at the research question, trying to see it yes with the quantitative research method, but trying to see it, the legal implications and the economic implications. If you look at the economics, they’ve made that jump maybe 50 years ago, if you look at political science they made that jump to quantitative research methods, what 30 years ago, 40 years ago, a long time ago. If you look at law we’re still in the bloodletting [laughs] age or type of research.

We certainly lack the ability to, most of us to do consistent high quality quantitative research using good empirical research methods. I suspect that the reason for this, and the real explanation for this is most of us, me included, do not have a quantitative background, i.e., we dropped maths in high school when we were 15 or 16, and that effectively makes us blind to that whole area of research and to those oldest methods that could help us actually look into legal questions from a legal perspective, not a legal economics perspective, but even from a legal perspective and reach conclusions using those methods. So I think I will finally have to take the plunge on that one and instead of just whining and complaining and moaning about it, actually try to do something about it, so that is certainly something that I will be jumping off a cliff pretty soon, I hope and suspect, what’s your view?


I think you’re right, and I think particularly in the way things are going with more interdisciplinary research and funding, really nudging people to try to go beyond their own discipline, we need to start doing more quantitative analysis and it’s going to be challenging to determine what can be done, what cannot be done, there’s also going to be lots of issues now around sort of automisation or artificial intelligence and how that can be applied to law, but I think that’s another area where definitely there’s lots of stuff to do in the future and, you know, as you say we need to just get out of the conference zone and go back to high school, if it’s about math or starting getting trained on coding, which are very different challenges and skills we need to develop, but that’s the future.

But I mean those problems were already faced by the other disciplines, they’ve gone through that, they’ve gone through their walk, to try to figure out and get out of the dark ages of research methods, so they’ve done that, so we can do that in law as well. Now there is one thing I would like to comment about, you’re right about the increasing pressure for interdisciplinary research and that is important because it gives us and everyone else a different perspective and a different view and frameset to interpret questions. But I still think, and will argue, that there is space as there is in economics and political science, to use research methods from those sciences, or imported from those sciences, to answer legal questions, which is slightly different from interdisciplinary research, and with this I don’t want to minimise the importance of interdisciplinary research and I think it’s very important and it’s key that more and more people are willing to work at the boundaries of their own disciplines, but for what concerns law, I think there’s space as well for legal research to be done by quantitative and empirical research methods.


You might be right, yes, but again another thing that is going to keep you busy for the future now [both laugh].

So is a life of an academic.


So it is. I mean you promised your audience a longer than regular podcast, I think they are now on the two and a half helpings of it, is there something else you would like to discuss or would you just want to draw this to a close?

I think we can draw this to a close, I was just wanted to thank some people before we finish, so first of all I would like to thank you for being the first guest of the podcast, and the last host of the current series, I don’t know if there’s going to be a third series or not, if there is going to a third series it’s going to be a probably a different format in terms of interviewing people from other areas and not just early career researchers, but at this moment in time I don’t think it’s going to happen any time soon, it may happen maybe next year, so big thank you for you Albert for both challenges at the beginning and at the end. Then a big thank you to all my other guests and interviewees that gave their time, gave their knowledge as well and were willing to be part of this project. And finally a big heartfelt thank you to the British Academy which has been absolutely incredible in the level of support that they’ve given me over the last two years. They’ve been very good in terms of the financial support, but also the ability to understand certain changes in circumstances, as life sometimes changes, and that’s why that has happened to me in the last year or so. So I’m very grateful for them, for the support, all the support that I’ve been given from the British Academy and the British Academy research officers  involved and particularly with the podcasts, so that’s pretty much it, it’s just a big heartfelt thank you.


And the same goes to you and I think that all of the public procurement community really appreciates the fantastic content that you have created over these three years, I think I speak with them just to encourage you to find a way of doing a third series some time, about something similar because this has been great fun and very enlightening to hear and to read and I think that there’s a lot of exchange of ideas and new networks coming off of this, so that’s very good work.

That’s very true, thank you very much, and just one final note to cap it up, I was incredibly surprised by the take-up of the podcast, there was a lot more people interested on these topics than I anticipated at the beginning, a lot more, so thank you for the audience, thank you to the audience as well for turning up every month or so to listen to the podcast. So that was the last Public Procurement Podcast episode for season two, you can find me at my blog, or on twitter where I use two handles, @Detig for general discussion, and @publicprocurement for public procurement related topics, as ever I’m incredibly grateful for the support of the British Academy Rising Star Engagement Award, which made possible this project. If you like the show it will be really helpful if you can rate it on iTunes helping others finding it where it’s going to be staying for the foreseeable future. Until next time, maybe next year.

#31 - Francesco Decarolis (Einaudi Institute for Economics and Finance)

Reputation and corruption in public procurement

Interview with Francesco Decarolis, Associate Professor at Einaudi Institute for Economics and Finance in Italy. Francesco is currently a Research Fellow of the National Bureau of Economic Research. He was recently awarded ERC Starting Grant to investigate reputation and corruption in public procurement.



Hello, Francesco, welcome to the podcast.

Hello, thank you for this invitation.


Francesco, I usually like to start the interviews by getting the guests to speak a little bit about their background. Could you tell us a little bit more about yourself?

Yes. So I have been working in the last 12 years in the USA, so as you were saying, I completed my PhD in Economics at the University of Chicago, where I focused on the analysis of auction and procurement markets, which is a rapidly growing and very successful area of economics, with a broad range of applications, from auctions and procurement systems that government use to various private sector applications.. Examples include spectrum, allocation for telephone operators, the allocation of oil and gas exploration permits, and really almost every other area you can think of. My origin is Italian, and while studying in the US, and learning about this new area of economics, I realised that there was a big discrepancy between many of the economic models and the practice. The actual way in which certain procurement, important procurement markets, were arranged, and that it was particularly interesting to try to bridge what was in our economic textbooks with what was happening in the practice, because these textbooks were missing some key features of the real-world environment. But also these real-world environments, and the rules under which they developed, were sort of addressing in an imperfect way, in a potentially problematic way, some of the real problems that they were facing. And so this is how I began to build a career as a researcher in the area of auctions and procurement. My studies are based mainly on the empirical analysis of auction and procurement data, but also with some focus on the underlying theory.


What brought you back to Italy, then?

So, in essence, the ERC grant.  I worked in several universities in the US. Most recently, I was at Boston University, in the Department of Economics, and while I was there, more precisely I was on sabbatical at Stanford University during the Fall 2015, I learned that I was the recipient of one of the ERC StartingGrants for economics. This grant is an absolutely fabulous opportunity to allow me to conduct research on areas of public procurement that I did not have the time or the resources to explore before, but that I always found very interesting and fascinating. Especially the use of past reputation in public procurement and the role of corruption in procurement. And the reason why I mentioned that I didn’t have the resources is also that for doing some of these studies, especially the one that I’ll be glad to tell you about in a few minutes, I collaborate closely with contracting authorities that implemented these rules, and working with the contracting authorities to try out different procurement formats, evaluating from an ex-ante perspective how to design the new auction formats, then doing the work of collecting the data, analysing, possibly having multiple meetings, both with the contracting authorities and with the suppliers, are very time consuming activities. And so thanks to the ERC grant, I have now all the financial means to perform these activities. Hence, I was very happy to relocate back to Italy. Right now, I am in the Einaudi Institute for Economics and Finance, which is an institute created by the Italian Central Bank to be an excellence in the study of Economics, and here in this institute, several important researchers for the area of public procurement have affiliations, as well. Giancarlo Spagnolo and Elisabetta Iossa for instance.


Very well. So, if you could drill a little bit into the details of the grant, and the research, what are you actually going to be looking at, more specifically?

The premise is that when we think about contract procurement, two features are absolutely crucial. The first is that it is unavoidable that there is some cost uncertainty at the time of bidding. So regardless of whether the procurement is a very complex contract, for instance, for the construction of new military equipment, or whether it is a contract for something easier like repaving a road, there is intrinsic uncertainty. Even in repaving a road, typically the bidding takes place several months before contract execution, and what the weather condition, what the cost condition will be at the time of the execution of the contract, cannot be known for certainty by the firms at the time of bidding. Second, coupled with this intrinsic uncertainty with contract procurement, there is typically a difficulty to verify ex-post performance and to eventually enforce penalty in all those cases where things have not followed the original contract specification. This is a feature that makes contract procurement very different from the type of auction markets that the economics literature has extensively analysed, and for which a lot of important results have been established. For instance, if the auction entails a transaction that will clear right after the auction, think about the auction for a painting, in which individuals or firms bid, but then the transaction clears immediately, then the kind of problems and the kind of solution are extremely different, relative to those of contract procurement, and in particular, in contract procurement, this issue of the ex-post life of the contracts becomes crucial, and makes competition - which is the typical tool that we see as being so important and so effective in auction - a double-edged sword. Things can go pretty bad if competition is pushed and exacerbated in an environment with cost uncertainty and difficulty to verify performance, ex-post, because firms that are unreliable are willing to offer a low price at the time of the auction, might not perform as they should afterwards.

So what does the private sector do in the face of this? Well, a series of things. It can require financial guarantees, like bonding, letters of credit, and it can embed the decision maker to use some form of discretion in selecting bidders. So discretion in the form of contract, or awarding method, negotiations vs auction, or also - and this is typically the case - discretion in the use of past performance. Just think of a very simple example: when you’re renovating your apartment, the past performance of the firm that work with will certainly play a key role in your choice, and this is widespread in the private sector. Now, what is remarkable about public procurement, and especially European public procurement, is that the use of past performance is strictly limited. To be more precise, until very recently, until the latest round of European directives, the use of past performance was strictly forbidden in Europe. And this is peculiar, not just because it makes the public sector so different, relative to the private sector, and potentially it limits a great tool to prevent the problems that I was mentioning at the beginning, of poor performance, but also because it puts the European system at odds with the US public procurement system. In the US, in a nutshell, since 1994, there was a major reform of the Federal Acquisition Regulation, that put past performance of contractor at the heart of the system for selecting suppliers in federal procurement, and the main idea was exactly that of mimicking the good practices of private sector. Now, Europe is gradually moving towards something similar, but very, very slowly, and we are still far away. So what I want to study with this piece of research that I am now conducting is to what extent this reform tried by the US, and other possible reforms, based on the role of past performance in public procurement, can be an effective way to combine improvements in performance together with, still, the objective of limiting prices. And in particular, what I wanted to understand, and study with this research, is how to combine the use of past performance within system of awardings based still on auctions, on transparent auctions, and in particular,  how to change from  price only auctions, or scoring rule auctions, that do not include reputation, to price plus reputation auctions. So scoring rule auctions that included reputation. So how to measure reputation and how to include reputation into the scoring rule, and then quantify how this matter, how this approach could impact both the performance delivered and the cost of this potential improved performance. So this was the key idea of this piece of research, and the implementation of it is something that is still ongoing, but I have some preliminary results that, if you are interested, that I am happy to discuss.


And we will do that in a minute. Before we get to that, I would like to go back to something that you said about taking into account past performance being strictly limited. Yes, and no. It’s true that you cannot, in EU procurement, use past performance to assess the quality of a tender. You can use it, however, to assess the quality of a tenderer in a previous moment to the actual bidding. So, for example, if you have a restricted procedure, if you use a restricted procedure, you can use, and you can allocate significant points, number of points, to the performance of the economic operators coming to the auction, or coming to the procedure. With the open procedure, it’s a little bit more complex, especially now with the changes that have been introduced, but it can also be done in a certain way. So, from my perspective, and obviously I’m coming from a legal background, so from my perspective, what you’re suggesting could fit within the current rules, and the current rules exist for a very good reason, or a very reasonable reason, at least, which is to ensure equal access to the markets, not only to companies with a huge trading history, but also to companies that do not have a lot of trading history with the public sector. So that’s why you have this clear separation, and this clear rule based on the principle of equality, to make sure that companies or that tenders, at the tender stage, are analysed on as much of an equal ground as possible.

I have two answers. One is about what the current rules say, but the second is even more important, because you have mentioned what I think is really the greatest departure in the view between economists and legal scholars. So let’s jump absolutely into this, and great that you asked. Why you said, essentially, that the rule to limit the use of past performance is there for a good reason? Because you want to keep an open market. Now, this is absolutely, absolutely at odds with how economists see the problem. Why? Don’t we care about open market and new firms and preferential treatment? Of course we do, but the issue is that you are sort of tying the hands of these public administration in a way that is not realistic. In a sense, whenever you are specifying how past performance shall be used in a contract, you must also specify how firms with little or no reputation must be treated. Now, the fact that you are using a system that is based on reputation does not necessarily mean that you are going to say, “We are going to use past performance for those who have, and for those that they don’t have it, they cannot enter the auction.” You can say something completely different. You can say, “We are going to use past performance for the firms that have a past performance, and for those that don’t have it, we are going to give the maximum points, or the minimum points, or the average points, or points estimated in a certain way.”  This is what we call a design issue. It’s completely in the hands of the designer., You can think of the designer as whoever writes the law, the rules of the game. So, what happens to firms with no reputation, to new entrants, to firms that also have, maybe, little reputation, is the result of a designer’s choice. And so it’s something that can be optimised and tailored to the market. The most important thing, the only message that I want to communicate is that you don’t have to see that there is a necessary block created for new firms from the use of past reputation, because it’s fully a designer choice, what happens to new firms. So tell me if this sounds convincing to you or not, because this is really something which seems to be misunderstood in many discussions about past performance, like those on the potential implementation of a reputation system in the European Directives, and I’ve seen this in the debate around Article 57.



In which the European Directives 24 2014 has introduced some use of past performance, along the lines you were mentioning at the beginning of your remark, and I see that the point that you mentioned, the criticism that you mentioned, was really prominent there. And tell me if I manage to be clear or not, because this is a key point. It’s really a designer’s issue. So things should be decoupled. We should first decide if we want to use past performance, because, if we think it’s important. Then we can argue, “What’s the best way to treat new firms?”


I gave you the opinion, the general opinion from legal scholars, and why the rules are why they are. In my opinion, as time goes on, I think that you are on the right track in the sense that there is a lack of reputation being actually taken into account in a good way. That doesn’t mean that it’s quite easy to do within the logic of the system, which is to ensure that both new entrants, and older entrants, are not discriminated against, because even with the design, and yes, it’s a design issue, even with the design, it may be very difficult, in practice, to actually making sure that the compensation given to the new firms actually does not create arbitrage, because if it makes life a lot easier for new entrants, then what’s going to happen is that the existing companies, or existing economic operators, will create shell companies to actually be as fresh entrants into the market.



There’s a risk then if you think about how the system works in general, and I’ve mentioned this in previous podcasts, the public procurement rules in Europe exist not to enable great procurement. They exist to avoid really, really bad procurement. So avoiding corruption, avoiding making stupid mistakes, those are two of the key objectives of procurement, not to be economically efficient, because if you’re designing a system to be economically efficient in terms of public procurement, we might end up with something very different from what we have now. So my perspective, and my personal view, is that yes, we should be moving towards a system where reputation is taken into account. So, about a year, or two years ago, I remember writing a blog post suggesting that perhaps we needed something like the eBay ratings system, or Uber, where both parties of a transaction, at the end of a transaction, are compelled to provide feedback on the counterparty, on some sort of exchange that is public, and that anyone can consult, and you end up with a registry of reputation for both the contracting authority and the supplier. So we actually address the reputation issue from both ends. I haven’t written anything more specific about it, but that was something that I was thinking about, because even for a supplier, it’s important to know how reliable the contracting authority is, because…

 Let me tell you a story, if you want, exactly on this. So in Italy, following this directive, and the implementation of Article 57, in the Italian Public Procurement Code, there has been a lot of debate, and essentially the authority in charge of supervising the public procurement sector, ANAC, Anti-Corruption Authority, put forward a proposed system to monitor past performance by suppliers, and at the heart of this system, there was essentially the contractual performance measured in terms of percentage delay in time of execution, relative to contractual time, and percentage discrepancy in final cost of the procurement, relative to the contractual price. Now, the Italian firms complained, and complained in a very reasonable way, saying that many times, it’s the fault of the public administration if things cannot be completed at the condition that was originally promised.


That’s true.

 And this was extremely reasonable, and I completely agree with this point, and in this sense, they were saying, “We need also a rating system for public administration, and we need something different.” So the quarrel around this proposed implementation of this part of the directives in Italy was so strong that the supervising authority ended up blocking these proposal.


I think that is a pain point that we could address in public procurement going forward. Focusing back on your research, you said that you had some preliminary data to talk about. Can you tell us a little bit more about it?

 Absolutely. So it all started with a very interesting experiment. Essentially, in Italy, we have a very large public utility company that provides water and electricity to Rome and central Italy. This company is owned 51% by the municipality of Rome, and so, because of this, it has to follow the public procurement regulations, but also since it’s a public utility company, it is in a group of contracting authorities the Italian public procurement code calls “special sectors,” and special sectors have some degree of freedom in how they design procurement rules. In how they fine-tune the procurement regulation that they apply. So the CEO of this company was extremely worried about the performance in the contracts that they were awarding. They award, every year, about 300 million Euros’ worth of contracts to perform maintenance of their water pipe system and electricity network, and with these contracts, they were considering the prices that they were getting from bidders in the auction quite good, but they thought that quality was really terrible, and you have to think that, especially for electricity, this is one of the really dangerous areas. If safety standards are not respected, people might die, because they work with high tension and you can easily get electrocuted, and the consequences are devastating. So what they did is they started to think about a system, or a way to improve performance, and then they consulted Giancarlo Spagnolo, and through Giancarlo also me, and we started a collaboration on this very fascinating project about how to improve the performance while still being leading within the system of the Public Procurement Court. And so if you think about introducing reputation, there are really two separate pieces. The first answers to the question: what/how do I monitor? How do I construct a measure of past performance, of reputation? And you can think that you could use publicly available measures. For instance, you can use some ratings that are already out there, I don’t know, ISO 9000 certification, or some other publicly available ratings, or you can construct your own rating measure. The company, with regard to this first question, decided that they wanted to build their internal rating system, and in particular, they said that they wanted to experiment two sectors. They picked electricity, and they picked two sub-sectors within electricity. Public illumination, and the maintenance of electrical substations, and they said, “For these contractual classes, we can write down a list, that is exhaustive, of all the things that need to be done properly in the contract, both in terms of safety parameters and in terms of quality parameters, and what we’ll do is, essentially, we’ll have teams of our engineers from the contracting authorities that then will go and inspect the execution of these contracts along this list of parameters.” And in particular, they chose a list of 136 parameters, and they also decided that these engineers that were going to do the inspections were randomly drawn. So their pool of engineers was changing every time, according to this random draw, to limit the risk of corruption. And also which work sites were inspected was also the result of a random draw. This is because it’s costly, it would be costly to monitor many, many contracts. By introducing a drawing system, you have a smart way to give to every contractor a positive probability of being monitored, but without having the cost of monitoring every contract.


Of course.

And so they sent these people, and they started to monitor, and the results were devastating, because the results of the first three months of monitoring revealed that noncompliance was overwhelming. So these parameters were scored with a zero if they were found to be noncompliant and a one, if compliant. When the scores arrived for the first three months and they were aggregated up, noncompliance was 75%. So only on 25% of the parameters, on average, they were doing something that was as written in the contract, and this was very bad, because again there were, like, safety measures that were violated, putting people at risk of very serious dangers, and this was across all contractors, across all types of works in these two categories, across all parameters. So it was a very widespread phenomenon and this was very much confirming the fear of this company, that noncompliance to contractual elements was very common. These were things for which in principle these firms should have been, or could have been brought to court. But this was not happening. These were things for which, in principle, penalties that were written down in these contracts could have been enforced, but this was not happening. And so, the reason why it’s not happening would open a separate chapter, it’s related to deficiency of the court system that, for instance, in Italy is quite bad, but it’s also related to the general phenomenon that in business, you try to maintain a good attitude with your suppliers, as well as with your customer. And so this firm was feeling that if you started to bring suppliers to court, and to enforce penalties, it would have acquired a bad reputation, and it would have been harder for this firm to conduct business. It was not doing this, it was not bringing them to court, it was not enforcing penalties, but it was very worried about this poor performance.

Moving to step two, after we have designed a reputation system and you have some numbers that you can use to quantify reputation, you have to decide how to incorporate these reputation measures into your decision of future awardings. Now, this is where things become tricky with respect to the regulation. As we were discussing before, under the current EU directives, you could use this information in the stage of selecting which firms are admitted to participate in the auction. Still, you could not use this to select among the bids that are received. So this firm was a little bit torn about how to use the reputation system that it had designed for the selection of future contracts. What people at this firm did was that they announced to their suppliers, they were going to switch to a scoring rule auction that was giving 75% weight to price, and 25% to past performance, where past performance was a weighted average of those zero/one scores that I was mentioning before. So the system clearly was living in a grey area between what it was allowed and what was not allowed by European directive, and this forced the firm to delay the implementation as they were collecting opinions from legal experts, but this made the experiment even more fascinating. Why? Because for a year and a half, basically, the suppliers of the firm had been informed about this intended switch, but the switch was not happening. Why is this so interesting? Because as we were discussing at the very beginning, and your main concern about openness of the system, if they were to switch immediately to the new scoring rule system, we would have observed, in the data, a mix of effects, in part coming from the selection of new firms that are kicked out by the new system, and in part, like, price and quality effects linked to the response to this new system. Instead, what we have here in this beautiful experiment is this one year and a half in which suppliers have already incorporated that eventually the scoring rule system will arrive, and so earning reputation matters, because if they earn reputation today, when the new system will arrive, they can offer higher price and still win the contract because they have earned a good reputation that will be valuable to earn points, in the scoring rule formula. But today, they still don’t face any limitation to their entry and to their probability of winning related to their past performance, because for the space of a year and a half, they still have a first price rule that is in place.

So this allowed us, essentially, to study the effect of the announcement of this switch to reputation, so not the actual switch, but the announcement of the switch, on the behaviour of the contractors in terms of quality delivered, and price, and the results were quite stunning. So basically, in a nutshell, I told you 25% compliance before. After one year post the first announcement in which suppliers were informed about the intended switch, and they were shown the scoring rule formula that was going to be in place, quality, overall, and performance, increased to about 80% compliance on the parameters. So from 25% to 80%, which was absolutely great, and the change involved, essentially, all parameters and all firms that were part of this market.


Has that maintained over time, and has it also affected new suppliers that came in after the announcement, so they were not actually in the market during the announcement?

It maintained over time, and this 80% has remained more or less flat since then. Now, in terms of the selection of suppliers, what is interesting is certainly that we see that even the suppliers that were not performing well before the announcement of the switch to reputation, changed their behaviour. What we learned from this experiment is that we don’t have, necessarily, to think of markets with firms that are intrinsically good and bad, but there is very much a possibility for firms to be responsive to different incentives and to change, their, for instance, managerial practices in how much they value safety, and safety practices, and so on, and so we saw a change that is across the whole spectrum of firms participating in these auctions. In terms of entry and exit, we saw an exit of some firms, but in a way that, when we tried to benchmark with what happened in terms of the auction for the exact same sectors, but taking place in other multi-utility companies in Italy, and the exit rate that we observed for this firm running the experiment is an exit rate that is quite similar to the one of the other multi-utility company that did not run any experiment like this. In terms of entry, we see that there is some entry, but not much, in the period that we observed. Since only three new firms entered, from a statistical perspective, when you have such low numbers, it’s difficult to draw conclusions, because they don’t have any statistical power. So I would say with reasonable certainty that there is a great effect on behaviour, with the same firms changing. There is no effect on exit of firms. There is potentially some effect on entry. Potentially limiting a little bit entry, but it’s not clear how much.


Okay, so we need to wrap up the interview, we’re reaching the time limit. What’s next for you? So what are the next steps with your research?

There is one part that is still related to this experiment, that is, in a second part of the experiment, we were able to relate this performance also to the prices, and we observed that prices, in the initial phase, after there was the switch, they declined, probably because firms were competing very intensively to win contracts, because only by winning a contract you could be monitored and you could earn reputation, and afterwards, prices started to increase, which is compatible with firms passing through this higher cost of performing better, of delivering higher quality. But the passing of cost was not major, in the sense that, compared to the great increase in quality from 25 to 80%, the increase in cost was about 7%, relative to the price they were paying before.


And you could argue that is actually not an increase in price, it’s the price that it should have been, originally, if the quality was supposed to be the one that is in the contract.

We cannot say this for sure. We are not able, as economists, to measure what would be the cost of this quality. From the perspective of the firm we work with, we know that the firm is happy. It thinks that this is a very reasonable price increase for the higher quality. What we have done is a sort of exercise of trying to quantify the welfare produced by this reform, in terms of lives saved, so reduced probability of accidents, and by weighting this reduced probability of accidents with the value of statistical lives, which is a quantity that economists and also social scientists often use, and we saw that, indeed, the policy produced great benefits. So the benefits exceeded the cost by an order of several million Euros per year. So all this suggests that the policy was positive, but we don’t want to stop here, we want to keep on studying and understanding this system. So what comes next is that this company is trying to now understand how to restructure its system to be compatible with the new directive, again, Article 57, as we were discussing, and so how to switch to a system in which the reputation index is used to select firms that will participate, not to select bidders, and it will be very interesting to see how these two completely opposite systems perform, which are the pros and cons, but in terms of prices paid, and in terms of quality delivered. And so this will be, we believe, a fascinating way to learn about different methods of regulating this system. Of course, there are many other questions, because it’s quite arbitrary. Why 75% on price and 25% on past performance? Why those specific measures of past performance? This is really a broad area, and we have several interesting projects, mostly with Giancarlo Spagnolo, that we are pursuing in this area. On a slightly, still connected, but different part of my ERC grant my current research focus, is on the area of corruption, and the problems of criminal infiltrations into public procurement auction, and these are projects with Ray Fisman and Paolo Pinotti.


Thank you very much, Francesco, it was a pleasure having you.

 Absolutely, same for me.

#30 - Niels Uenk (Public Procurement Research Centre)

The challenges arising from procuring long term social services

Interview with Niels Uenk, part-time Researcher at the Public Procurement Research Centre a joint interdisciplinary research centre of the universities of Utrecht and Twente. Before joining academia Niels worked 5 years as international consultant in supply chain and logistics optimization. He specialises in public procurement of long term / social health care services.



Hello Niels, welcome to the programme.

Good morning Pedro, thank you for providing me the opportunity to talk about my research.

My pleasure, so Niels you have a mixed background because you’re working in academia but you’re still connected to practice, so can you give us a little bit of a flavour of what’s your actual background?

Yes, so after graduating my Masters in industrial engineering and management I worked five year in a private company called ORTEC like you mentioned in supply chain and logistics optimisation, and after five years I was ready for a new experience and I joined the Public Procurement Research Centre which has quite a unique position for me. Half of the time I get to work on my PhD research and the other half of the time I do paid work, so research and advisory work for other institutions, and basically to finance my PhD and that allows me to do my scientific research. Most of the other work I do is actually in the same area, so focusing on the procurement of social care services.

And have you been doing that for long?

I’ve been doing that since, well late 2013, starting my research around early 2014 actually.

Moving onto the actual research that you’re doing, can you provide us with a little bit more detail what are you actually doing your work on?

Yeah, and I’ll pay a bit attention to the context in the Netherlands as well, so my research focuses on the procurement of social care services, and in healthcare procurement we distinguish between two types: you have procurement for healthcare which actually boils down to buying anything from rubber gloves to equipment that is necessary in the process of providing healthcare, for example a buyer in a hospital, that is on the one hand, and on the other hand we identify procurement of healthcare, so actually buying the healthcare services often from healthcare insurer point of view, or from a public body’s point of view, buying or commissioning healthcare services. I strictly focus on the last part, and it’s a very relevant topic, I focus on the long-term care services, for example home care and a lot of care services that are not medical in nature, so it’s not fixing broken bones, or doing surgery, but it’s rather the type of care services that allow people with a disability or people that come of age to keep living on their own, to participate in our society, so it’s for example mental therapy for people with anxiety disorders, or also sometimes pretty straightforward assisting elderly with doing their tasks around the house, cleaning their house, and in doing so it prevents those people from having to move to an institutional home for the elderly. Especially in developed nations, the expenditure on these kind of services is rapidly growing and it’s one of the main concerns and challenges in healthcare for a lot of developed nations as well that the expenses and the expenditure is quickly growing because of for example the greying society. So in the Netherlands for example we spend about 15 billion euros a year on these kind of services, both for adults and for youth and it’s actually growing at a rate of 5% or 6% a year, much faster than our economy is growing, so that provides a challenge.A lot of other countries, the UK, I think the most countries in the OECD face similar challenges.

That is true and it’s certainly the case here in UK.

Yeah, in my research context what is very interesting is that in 2015 in order to do something against these rising costs and also to improve the quality, in the Netherlands we underwent a massive system reform where these services were previously centrally coordinated and centrally arranged (and procured), responsibilities switched, moved to local governments, so the municipalities, we have about 400 in the Netherlands, from 2015 became responsible for arranging and thereby also procuring these social care services. It was one of the biggest system reforms in the last I think 60 or 70 years, the budget involved like I mentioned is about 15 billion euros and the municipalities got a lot of freedom to actually manage these services to their best insight. That creates a very interesting situation where a lot of municipalities are choosing different ways of commissioning these services, choosing different ways of paying for these services, they are designing their own system of how people should apply for these services and a lot of different approaches are visible right now which for me as a researcher is obviously a very interesting position because I actually get to, on the one hand investigate how municipalities are approaching the procurement and the commissioning of these services, and then see if the difference between those approaches actually results in better or worse care for the citizens. So that is basically the main topic of my research, how are there municipalities contracting this care and does a different approach need to improve quality, reduce costs, etc.

It’s interesting because my impression here in the UK is that long-term care services are procured by local authorities separately from healthcare, so and one of the discussions that I’ve seen over the years popping up time and time again is actually merging long-term care or social care with healthcare procurement, because at least in UK there’s the view that by keeping them separate each one of those two systems tries to shift patients to the other system so that they don’t have the cost.


So what’s the experience in the Netherlands if it’s possible to find any kind of information about it right now?

One of the reasons for decentralising long-term care to municipalities is in fact to try and bundle a lot of responsibility within this social care sector with the same organisation, so that the municipalities have a very wide responsibility right now for any kind of support, social support, social services that may be necessary within the situation of a client, so before for example the social care for youth was very fragmented in the Netherlands, where provinces had responsibility for one part, regional offices of healthcare insurance agencies were responsible for a different part, and the local government and the central government was responsible for a third part and all these different parts have now been combined and made the responsibility of the municipality. Indeed there is still a distinction between the private, the curative healthcare procurement, so the procurement of hospital care and basically any type of care that actually involves medical care, (and the social health care procurement, red.) there’s still a division, and there is still also debate on whether certain types of care will be decentralised in the future as well, in order to optimise and be able to manage as integrative care system as possible. But for now these types of social and home care, as they are quite different from the medical care you receive in the hospital, the municipality already have quite a lot of responsibilities as they have right now.

Moving onto the research that you have actually done so far within your PhD, as far as I know you’re doing it in a similar way to one of my previous guests, Suvi Taponen, you’re basically doing your PhD by publications, am I correct?

Yeah, that’s correct.

So what have you been doing so far and what have you concluded in terms of research?

One of the first things I started to do, I started right in the year 2014 when all the municipalities have had to contract these homecare services, and what I did was, well I wanted to find out to what extent these municipalities took different approaches with respect to procurement. They all came from the same situation: before 2015, the procurement was standardised for the entire country, the same products, the same tariff structure, the same procurement procedures were applied. The first thing I did when I started was to collect the procurement documents and tender procedures and any kind of information on how municipalities actually procured these services, for as many municipalities as possible. So I started making a database with all the tender documents, all the contracts I could find, and most of the municipalities actually published these services, or these contracts which in 2014, was not actually required from public procurement regulations because these services are actually back then part of appendix 2B-services (EU regulations on public procurement, red.) that were exempt from most of the public procurement regulations. The municipalities, most of them, published their contracts anyway and I made a database of that and the database I have has about 95% coverage of all these tender documents of all the municipalities, so there are approximately 400 municipalities, I have the tender documents about 385.

That’s a big dataset.

That’s quite a big dataset, I have to say thank god for me most of the municipalities actually also collaborated, so it actually boiled down to 90 different tenders, still it’s quite a big dataset because most tenders, well if you have seen public tenders before you know they can contain quite a lot of different documents and quite a lot of extensive documents.

The next thing I did was, well to actually build the database with analysis of these tender documents, of well about 20 different factors ranging from aspects of the procurement process: did the municipality collaborate with the procurements, how big was the collaboration in size, how many different municipalities joined, what kind of procedure did they follow and then there was a lot of negotiation going on and a lot of dialogue in those procedures, was the procedure open for any kind of care supplier or did the municipality invite a selection. I also analysed the content of the contracts: what kind of products did they procure, did they simply copy paste the product lists that was used before 2015 or did they create new products, what kind of reimbursement method did they apply. So I analysed all those tenders based on all these factors, also what kind of tariffs did they pay and I created one quite big database based on the procurement approach, each individual municipality or collaboration took, and it is actuallynow my starting point to also write scientific papers based on this dataset. So I have the first paper out, it was just yesterday accepted at the IPSERA conference, so I’m very pleased with that as a competitive paper, where I will present the results. In the paper I first do an analysis on what kind of aspects are relevant to study based on literature (on service triads, red.) and I then present the approaches of the Dutch municipalities, how they went about with respect to the relevant aspects.

You’re still working in practice so you’re just doing the research part-time and working at the Public Procurement Research Centre in a part-time basis, so from your perspective what are the challenges of managing this joint or dual track kind of life, managing research and practice side-by-side?

Let me highlight first the good things about it and then the challenges, obviously or what I really like about the joint work is I get to apply the knowledge that I find from the research and I get to directly put that in practice, or make use of that in practice, so more and more municipalities are inviting me to come and talk about my research, they want to know how do other municipalities tackle certain problems, they want to have an overview of different approaches that may be possible and well because I analysed that for my research I have a nice overview and I can directly put that into practice. Also, sometimes municipalities or also care providers, well they have like quite practical problems or issues they deal with, and it also keeps my research relevant to practice, because I’m not just doing some very theoretical research, because I know what kind of issues the practitioners face, I can also know what kind of relevant research questions I should focus on to actually be able to do research that is directly relevant not only for the scientific community and for our knowledge, but also for practice. Now that’s what I very much like about my position, one of the main challenges and I think every part-time researcher will provide the same answer is that for my work related activities there’s always something, there’s always a deadline and there’s always something waiting for an answer. And for my PhD that may also be the case, but the deadline is always a bit more soft, or if I don’t do the work this week well nobody is going to shout on the phone to me that I haven’t... so one of the big challenges is actually to on a weekly basis make sure that I also spend enough time on research, and yeah, I don’t have a very strict planning that the university keeps me to from when I should finish this article and when I should finish that article or that paper, but that actually increases the risk of not doing enough research and letting your agenda be dominated by the work related activities.

It’s interesting that you’re highlighting that because I felt exactly the same thing when I was doing my Masters, maybe 12 13 years ago, and I was working as a lawyer at the time and I just could not pull it off, exactly because what you’ve said, because if you have work related commitments they tend to have very tight turnaround timescales attached to them and certainly as a lawyer that was the case, there’s always stuff that needed to be done in that day, so you keep on pushing out for later and later those deadlines that are not as strict, and it’s almost like an adult version of the marshmallow test that it’s done with kids, so that if we delay gratification and actually work on those softer deadlines we’re going to be better off on the long-term instead of just fighting fires every day. But I could not pull it off so I ended up actually once my decision was made to do a PhD in actually moving full-time and I never considered even the possibility of doing it part-time as you’re doing so I have a huge respect for the researchers that are able to do those two different jobs and roles at exactly the same time because I find that incredibly incredibly challenging. Now one of the things that I really liked when I was a PhD student is that my supervisor was very, I wouldn’t say strict, but very assertive with deadlines, and that is something that over the years once I became PhD supervisor myself as well I found to appreciate more and more, which is it’s important to give PhD candidates a certain structure and a certain timetable that builds on them a certain rota or a certain way of doing work so that they can expect to submit a piece of work, be it a paper, be it a chapter, half a chapter, 5,000 words, whatever, with the certain cadance, i.e., a certain number of words, a certain number of pages every four weeks or every six weeks because that creates the habit of producing work constantly. And that worked really well for me as a student and it works well for most of my PhD students as well, so sometimes I think, and I have these discussions with colleagues, that we sometimes leave PhD students to their own devices and say, okay, they’re doing a PhD so they’re supposed to be autonomous, it’s up for them to manage their own time. And that is sometimes very hard and especially I would say it’s doubly hard for people like yourself when you have other competing interests, in this case actually a part-time job elsewhere.

Yeah, it is for me one of the main challenges, and having worked in a consultancy firm five years before starting my PhD I think actually helps me to have the, to make sure…

The discipline?

Yeah, the discipline, that’s what I was going, to have the discipline to make these deadlines for myself, although yeah like I just mentioned it remains being challenging, I don’t have a supervisor that’s strict with me and if I would have it probably would help, would help as well.

Okay, I’ll send him an email saying that!

Yeah, but I try to do this, I’m collaborating also on a few papers from time-to-time with others and I try to make deadlines on myself if I have agreed to work on something and to have it finished with a colleague it helps me because I hate to be late or to not manage that deadline. So I I visit a lot of writing workshops through my contacts at IPSERA, I get invited to writing workshops and I tend to use those workshops and those days as deadlines to just make sure that I have something done, have a new version or concept version of a working paper done, and working from conference to workshop to, well those are actually the deadlines that make sure that I, well writing enough. But to be honest right now for me it becomes much more important than in the first year because in the first year I did the collection of that data and it was both for the PhD but also for some external projects it was relevant, so up till now I’ve been able to combine a lot of the research activities with my activities in practice, but the writing of scientific journal articles, well that’s purely for my PhD, so I’m much more in the phase where I have to produce those papers where a lot of the data is already collected, so it actually becomes much more important in this stage of my research to keep those deadlines and to keep in what we mentioned, a certain cadance of producing scientific articles for book chapters, compared to 2015 and early 2016.

What do you see yourself doing after you finish your Ph.D, do you want to stay connected with academia, do you want to go back full-time to practice?

To be honest I don’t know yet, I very much like the position that we are in to be able to do really scientific research from university point of view producing to sometimes discuss about a research and the methods used with colleagues and to be able to, well improve the quality of research compared to some of the reports written by other firms, sometimes consultancy firms that sometimes just need to quickly produce something with maybe a more commercial motivation, so I really like the scientific base where we come from. But yeah of course if I want to stay in science and in academia writing and producing there’s much more emphasis on that (getting published) after the PhD maybe, so to be honest I'm not sure yet.

Very well, I think it’s a great way to finish the interview. Niels, thank you very much for coming.

Yeah, it was a pleasure, I feel like I haven’t discussed a lot in detail of some of the stuff I’m doing but maybe that’s also not for the greater audience, a bit less relevant, but I thank you for the opportunity and well I think it’s a very interesting way of sharing research with the world, so thank you and very interesting project you have.


#29 - Suvi Taponen (Aalto University School of Business)

Risk management in outsourcing of health services

Interview with Suvi Taponen MPA, a Doctoral Researcher at Aalto University School of Business, who also works as a Procurement Consultant. She will defend her thesis entitled ‘Improving the efficiency of public service delivery through outsourcing and management’ in the beginning of 2017, and she has worked previously at Hansel, the Finnish centralised purchasing body. The conversation is focused on the management of outsourced healthcare services, the core of her research.


Hello Suvi. Welcome to the programme.

Oh hello Pedro. Thank you.


Thank you very much for accepting to be interviewed. I really like to get as many interviewees as possible from various backgrounds, so could you describe a little bit what is your background please?

Oh yeah, sure. And thank you for asking me to participate. It’s really a great honour. I’m happy to speak with you. My educational background is in public administration so while I was doing my Masters I studied public management and also public law, and in my Masters thesis I got involved with public procurement. I guess that’s the initial reason why I ended up here where I am at the moment. And after graduating as Pedro you mentioned, I was employed by the Finnish central purchasing body for the government and I worked with Hansel for four years. In the beginning I was an assistant responsible for contract management and after one year I transferred to a sourcing consultant’s role which meant that I was then, I wasn’t working with framework agreements that much anymore, I was consulting individual procurement that the Finnish ministries and other organisations such as the universities that are funded by the government were doing. At the time I mostly worked with IT purchasing but also quite a lot with services. And after two or three years of practising I kind of felt like well I know there’s quite a lot of inefficiencies, the way that the government organisations operated, not only within their procurement functions but also mainly with how they were purchasing and why they ended up purchasing and at what stage of their planning of the year or planning of their service delivery. And that’s how I ended up considering taking up a PhD and well that’s almost an accident. I just dropped a casual email Aalto University School of Business and then I was connected with the right person who is now my supervisor. At that moment I wasn’t to be honest that serious about it, I was, and then, well then actually Professor Katri Kauppi asked me to come over and then we ended up talking and we definitely shared the same ideas and also shared the same ideas about how the public sector should be developed, especially in the Finnish context.

So I started out my PhD as a hobby, as crazy as it sounds. I was, in 2014 I was still working as a full-time consultant for that centralised purchasing body and then I was doing my research on the weekends. I already had the idea of starting off with looking at the make-or-buy decisions but I was really interested in the reasons of why public sector is outsourcing its services and what are the reasons behind it and is it always sensible to do that?

So that was the main motivation for my PhD because in Finland maybe you wouldn’t think that as we have been one of these traditional welfare states but outsourcing public services has been a really popular trend for maybe almost twenty years now. So it was really interesting in that phenomenon that what are the reasons and it felt like, and to be honest it still feels like that the outsourcing decisions are too much based on ideology and political agendas. So that was the main motivation for my whole PhD, directly into that issue, the outsourcing of services in the public sector context.


Did you think at the time it was important to answer that question or look into that question?

Oh yes. Because there were quite a few, of course the papers love to cover, especially the cases when outsourcing goes wrong, and it seemed like there weren’t too many facts presented in relation to analysing the results of outsourcing. So I had this doubt in my mind that maybe outsourcing, yeah,I think I kind of changed my mind, but in the beginning I thought that outsourcing maybe should be used more, that maybe public services should be outsourced more, they were outsourced at the time but then I realised that I need to look into the facts and provide some proof that would support a statement.


That is a very good point because it’s part of the process of doing a PhD is that we always carry with us our preconceptions and say “oh we think we’re going to find x” or this is what you’re going to be arguing at the end. And based on my own experience and also my experience of supervising PhD students, that often is not the case in reality, is that you actually changed your perceptions as you do the PhD, which shows that you’re actually doing a PhD and you’re just not trying to find only the sources that confirm your pre-existing bias?

Mm. Yeah, you’re definitely right. It’s you always have these ideas that you think that you’re going to be able to prove. But I think, yeah, I agree with you definitely that it’s extremely important and an essential part of the PhD process to accept that facts aren’t like you hopefully thought they would be and accept that, you know.


So you decided to do that research into outsourcing. What methods did you deploy in the PhD?

Mm, yeah. My PhD it’s an article-based PhD so it means that actually within one PhD thesis I have done four individual pieces of research. So they’re all individual research projects with different kind of methods and they kind of go through the outsourcing phase. So I started off with make-or-buy decisions as I mentioned and that’s a qualitative case study with a comparative perspective. So what I did, I had eight cases in total, four of them were from the public sector side and four of them were from the private sector, and well looking back now being a bit critical I noticed that maybe I should have included a bit more organisations but the small number is explained by the fact that as I wanted to compare the make-or-buy decision processes, like what triggered the process and what kind of criteria was used, I recognised that I need to have service deliveries that are actually identical in both sectors and those kind of services weren’t that easy to identify. And then in the end I went with occupational healthcare, translation services, leasing services, so that’s funding and also taking care of all the IT equipment and then customer services. So I was only able to get those kind of cases in Finland and four of those services.

So then what I did, I collected interview data from each case organisation and also used some written material I was able to get from some of the organisations which for instance described their decision making processes, if they had a model for that and so on. That was the first paper.


So that was the first paper that you did, so you did a qualitative case study…



and took a comparative perspective? Well I mean as you’ve highlighted it’s important to make sure that you can actually compare and be sure that you’re comparing apples to apples and not apples to oranges because otherwise you’re screwing up the data. So that was the first one you did. What did you do after this qualitative case study?

Yeah. Well actually I used that same data for the second paper or second article of my PhD as well. So at the same time when I was collecting this data I also asked the interviewers in the organisations about the transition phase because in all of these cases, eight cases, the result was to outsource all of the service delivery or outsource a part of that. So then also in the interviews I discussed with them about management of the transition phase and also contract management. So then the second article of my PhD which is actually also my first publication that is about to come up, it’s already available online which is really exciting, so yeah. So this second article was also a comparative study between the public and private sector but the perspective of the comparison was a bit different, as in the first paper the comparison is about identifying best practices from both sectors and, well we’ve been resubmitting the paper quite a few times, in some versions we are actually offering the recommendations for both public and private organisations but in the second paper it’s all about public service management.

So then in that paper the perspective is to use the private sector’s data to identify best practices that public sector organisations could absorb and implement. And I think looking back now that was a bit more, or is a bit more successful perspective as I recognise that in many of these ideas that I’m looking into, especially the make-or-buy decision, it’s really largely and already covered in the private sector’s specific research context. So the article is really, there’s practically nothing new if you put that in the private sector context but then if you especially try to make contribution to the public sector side, then it might provide more novel contributions.


On that note, I mean one of the things I find important to always take into account when you’re looking at private sector practice and how it could be transferred into public sector is how do you account and how do you control for the differences in terms of decision making and especially in terms of incentives between the two kind of organisations?

Yeah. Especially in the public service management paper we recognise the fact that it’s an issue that private sector organisations that are able to incentivise, provide incentive, monetary incentive, so promise more salary, higher salary if a person delivers better outcomes, but at least in Finland the contracts that the public sector employees have and what the unions have negotiated on behalf of the employees and also the employers is that they are not able to offer a similar kind of contract to their employees and incentivise achieving their outcomes for the service delivery.. And actually in some cases this has led to outsourcing services to private sector so the flexibility that they have in their contracts with their employees, so definitely you’re onto something when you bring that up.


Okay. So that was the second paper. Can you talk briefly about the next two please?

Yeah, sure. So then moving onto the third paper and it’s actually, yeah, that was a good thing that you brought up the specifics of the public sector organisations and how can you, how you cannot actually compare them entirely, the private sector, this was really highlighted in the third paper of my thesis which is about the risks of outsourcing a health service function.

So just to provide a brief background about that, in Finland it has been a huge trend to outsource. Some of the municipalities that at the moment still are responsible for delivering healthcare, they have actually outsourced all the services that they are responsible for providing to private organisations. So there has been a big trend of this total outsourcing in relation to healthcare. So my third paper I have, it’s a single case study from a medium sized city in Finland which has, the city has twelve outpatient clinics in which it provides primary healthcare and the city had already outsourced two of these outpatient clinics. And then actually from a private health service provider I got the information that now they are thinking about outsourcing more of these outpatient clinics.

In the third paper I combined, well it’s not a, I shouldn’t call it a survey but what I did I interviewed their whole project team that was preparing the invitation to tender and the suggestion from the City Council how the outsourcing should go about. At first I sent them a questionnaire and they all listed what kind of risks they see involved in this outsourcing idea that they were planning on and they gave values to these risks, two kind of values on the Likert scale, 1 to 5, how likely is the risk to realise and how bad are the outcomes, or what’s the impact of the outcomes if the risk realised? And after the interviewees had filled these blanks then I did interviews with all of them. And in the interviews we discussed the risks that they have identified and then we also talked about the state of risk management at the moment within the city and they also shared some experiences that they had from the previous outsourced outpatient clinics that they had. And actually my idea was to do a bit of a longitudinal study but what happened I was able to publish that paper in relation to the IPPC7 conference last summer. So it ended up being a bit, the scope wasn’t as wide as I figured in the beginning but it’s actually quite nice that then I had all the, as you have a limited amount of characters for the publications I was able to actually focus on what are the risks in the outsourcing consideration phase and how the contracting authority should mitigate these risks during the quantitative tendering procedure.

And now as my thesis is under pre-examination I’m actually continuing this research. So I went back a year after the interviews with the city and its employees and asked how the risks had developed within the year and what was the result of the tendering process, and now I’m actually, it is pretty exciting just Tuesday this week I also did interviews with the selected supplier so they are now also sharing with me the perspective on risks that they had while they were preparing the tender and now, and when the service is actually already running so from the transition phase.


And is it working well?

Yes. The results are good but actually doing the tendering process they, we identified that there’s a political risk involved as I mentioned that the final outsourcing decision is made by the City Council in Finland in the local government level. So what it means that the civil servants they’re preparing the suggestion and what they suggested was that they should outsource two outpatient clinics but what the City Council ended up deciding is that they only outsource one. And now looking back the civil servants still were, there were a bit heated discussions when they looked back to the decision because they really thought that the politicians were, there is an election coming up next year, yeah, so they were really frustrated with the fact that they felt like the election had more impact on the decision that the facts that they provided so…


That’s a great example of the differences between dealing with private sector and public sector?

Yes, definitely. So they’re really happy with the one now, the outcome of the outsourcing in terms of the one outpatient clinic, but the problems that were driving outsourcing and triggered the consideration they still exist within the outpatient clinics because obviously the politicians didn’t actually have the means to improve the state of the service delivery internally.


Okay. And what about the final publication, the final article?

Oh yeah, the final article I actually did with, that wasn’t, all of these I did while I was working at the Aalto University School of Business but last spring I spent at King’s College Policy Institute in London. So my final article is in the UK context and in the context of National Health Services, so NHS, and that’s about looking at the purchaser provider relations after outsourcing has been done, so it’s looking at contract management. And it’s, we had the, yeah, I did that with Saba Hinrichs-Krapels from at King’s College and what we looked at was how to cooperatively develop care, the outcomes of care, then you have outsource delivery, and that was also a qualitative case study. We had, in England there are these Clinical Commissioning Groups that are responsible for purchasing care and I think there are more than two hundred of those, so from the two hundred we were able to get twelve organisations that gave us interviews and in addition to interviewing the representatives of the Clinical Commissioning Groups we also got a few interviews of these strategic organisations that work nationally to improve care.

And the process of case selection in this, my final paper was actually quite interesting. It wasn’t as easy as it was before, as in actually not too many people are working with public sector in Finland and of course there is quite a lot of research going on but not that much research as in the UK context and especially the NHS, I think they’re a bit sick with researchers and all these organisations collecting data to improve their operations to be honest. So as in Finland contacting case organisations basically were like just giving them a call, introducing myself and after that they were happy to welcome me to give the interviews, it wasn’t as straightforward in the UK context. So we did quite a few rounds of emails and we struggled a bit getting case organisations but then we decided that okay, maybe we should consider only to one service, use one service as an example and then maybe we are better able to identify potential case organisations more efficiently and then also we realised that it will make identifying, achieving the goals of our research of identifying the means for cooperative development easier.

In the beginning I was only able to get two interviews and based on those we identified that cancer care is a really topical thing in the UK, topical in the sense that there has been a national programme of improving the care for cancer, there are huge variations in the quality of care…


That is true, yes.

Yeah. So that drives improvement of the care. So a lot of the Clinical Commissioning Groups were focusing on cancer care at the moment. And then I mentioned that there are these strategic networks that focus on three care areas, one of which is cancer care, so by contacting those organisations we were able to get a list of potential interviews, interviewees and organisations and actually I think one of the people working in these strategic networks actually requested them to participate in the research for us and that really helped. And then in the end I think I did maybe sixteen interviews which was enough to get, to be able to draw conclusions. But definitely was a tricky process which demanded a lot of persistence and patience to get all the interviews.


And in terms of findings, what can you tell us in terms of your main findings from all your research then over the last few years?

Yeah. It was definitely a big challenge when I was finalising my PhD to draw all of this together. I thought it would be a bit easier. But I think the main finding if I start from the beginning is that in terms of outsourcing triggers definitely private organisations they’re proactive. They kind of analysis the service delivery all the time continuously, well not every day of course but at least every quarter or twice a year. But what public sector organisations they typically outsourced when they have problems that have already realised in their service delivery, so they don’t really do it proactively. Of course there are some exceptions but the typical cases that the service availability is so poor or the service quality is so poor that they absolutely need to do something.


Why is that? Why is private sector much more proactive whereas public sector tends to be more reactive? Did you find anything, any justifications or explanations for that fact?

I think, well now speaking in the Finnish context, I think in Finland we actually, our public sector has been quite wealthy for quite a few years so they haven’t been until now, the scarcity of public funds has been a reality for quite a few years but before there was quite a lot of tax money to use for the operations and for the service delivery. So there wasn’t this demand of organising operations really cost efficienciently or that effectively in terms of quality. But then I think within the last ten years there has been less and less money to use for the service delivery and that has created a demand to deliver care with better impact with lower costs, as in if you think about private organisations they wouldn’t exist if they are not financially sustainable and cost efficient so…


So that brings us back to the question about incentives then, how the incentives play differently between public sector and private sector leading to different outcomes and strategies?

Yeah. So basically from the first paper we established that considering outsourcing makes sense if the service demand varies a lot. So then of course it’s hard to figure out how many employees you should employ in-house so it’s easier to outsource a service, but the demand varies so out to a supplier who provides that service to several clients. So that was one of the main findings that in that situation you should consider outsourcing. And then we also mapped the process of how you should go about the consideration.


Okay. So I think we’re getting close to the end of our interview. What do you think you’re going to be doing next? What are your next steps in terms of research that you’d like to turn your attention to?

Oh yeah. Well in the last two articles of my PhD I focused on healthcare, so I think that’s the main focus area. And then one of the reasons is also that Finland is currently in the political process of reforming the delivery of healthcare, so based on the piece of research I did in the UK I also identified the organisational structure and analysed that and contributed to the political process a bit by providing a report. So I think that’s something I’m going to continue working with.

I’m also now employed as a Consultant so I’m already returning to practice but I am hoping to continue the line of research with health services that I’m doing. So as I mentioned I have some more data on this risk paper so I’m going to continue developing models on public sector risk management, especially in the context of health service outsourcing. And then, well let’s see how, I’m not quite sure after a few years how I’m able to combine practice and research but hopefully well because I enjoy them both, so yeah. I’m a firm believer that a hybrid career is possible but let’s see.


Yeah. I find it interesting because one of the things I miss being full-time in academia is actually working in practical problems and solving practical problems, so it’s no surprise that when I actually do research most of my research tends to have a practical focus or a practical emphasis because it’s just me being pulled back to my origins as a lawyer and not someone that has made all his career in academia. So Suvi, thank you very much for taking the time to be with us today.

Thank you.

#27 - Ignacio Herrera Anchustegui (Bergen Centre for Competition Law and Economics)

Issues with the centralisation of procurement


Interview with Ignacio Herrera Anchustegui, Ph.D candidate at University of Bergen and member of the Bergen Centre for Competition Law and Economics. Ignacio has been awarded a coveted international Scholar-in-Residence program from the American Bar and significant funding from Statoil for a post-doc. His research interests are related to the interaction of public procurement and competition law in particular buyer power in centralised purchasing. 





Hello, Pedro, thank you very much for the invitation, I’m a big fan of the series and it’s a great honour for me to be here and have a word with you and hopefully say something interesting about the interaction between public procurement and competition law, also especially in centralised purchasing which is the area that I’m most interested in.


It’s a pleasure because I had you in the list for a long time and I was just waiting until you’ve actually submitted your PhD to do the invitation.

Yes, thank you, that actually was last week so looking forward to the evaluation and see if I pass or not but we will see.


Most of your research and most of your work is actually in competition law or in competition in economics so how did you end up looking at centralisation of public procurement?

Well, that’s a very good question because originally my PhD topic was going to be centralisation in public procurement or buyer power in public procurement settings and originally that was the starting point but it actually faded away because my supervisors thought that the topic was very complicated which I think it is, and it ended up being purely buyer power and anti-trusts or EU competition law. So then the main reason why I ended up in this question was because I come from a country, that is Venezuela, probably linked to the fact that we have a lot of money but the money’s probably not well spent in the public procurement markets and then I was thinking, ok, what do we do from the perspective of a public buyer when we have a lot of capacity and resources to enter into a market and how does this public buyer, or buyer in general, behaves in the market and can create problems but at the same time can do things that are a positive outcome. So the way that I ended up doing also centralisation of procurement has to do with the fact that the University of Bergen, I am the person that is in charge of the course on EU and EA public procurement law so it seemed like a natural fit and then as I delved into the analysis of buyer power economics and buyer power anti-trust regulations or EU competition law regulation. Then it shifted away from being the centre of my PhD thesis to kind of like my pet topic on the side and what I do on my free time, so to speak.


Yes, that’s make very much sense but if you’re doing it on the side and it’s a more complex topic, what have you been able to achieve so far in terms of research on it?

Well, quite a few things actually because I’ve been looking at what is the situation, I mean, firstly I’ve been looking at what buyer power is from a general perspective and that doesn’t change too much between what happens in a public market and what happens in a private market. But when it comes to centralised purchasing and when it comes to public procurement, my interest was seeing how is public buyer power created and how has it been integrated or enhanced, for example, in the EU directives and one of the things I saw is that when I started my project, the EU public procurement directives were not yet in force and weren’t even still a project, they were not even approved and I could follow a little bit that there was a transition from, “Well, we’re creating centralised purchasing agencies,” and this is like a pilot project that happened in 2004, forwards now to 2014-24, centralised procurement seems to be a very in-fashion tool to generate a lot of savings. So it ended up being kind of a side project but at the same time very important project for me because I was able to, at the same time that I worked on my anti-trust regulation, I could also see the similar issues that happen in private markets tend to replicate in public markets. Although, at the same time, public markets are slightly different because the buyer doesn’t behave in the same way as it does in a private setting. So it ended up being a very interesting and enriching situation in which what I learned from a setting that is purely private in the terms of competition law, retro activated or helped me understanding what was going on in the public side and vice versa. So I think the combination is there and some of the problems and some of the benefits, you can see them replicated. Of course it’s like modifications depending on what kind of sector you’re talking about and what kind of buyer power is being exercised.


Can you give us an example of those similarities and differences between the private sector and the public sector?

Yes, of course I can give you, what is the most important similarity? The most important similarity is buyer power; it is the ability of a buyer that wants to acquire something to reduce the purchasing price he pays for whatever he wants to acquire. And this happens both in the public sector and this happens in the private sector and one of the main reasons why this happens and this is very clear in centralised purchasing in public procurement is that you’re able to generate economies of scale and economies of scale in very simple terms, not to be very jargon, means that the more things I buy, the cheaper is the price that I’m going to get, right? And this happens both in the private setting or in a public setting. In a public setting a centralised purchasing agency is going to pool the demand of different contracting authorities and it’s going to buy either for all of them or on behalf of all of them in order to increase the amount of buying ability that it has and, therefore, by purchasing more, it’s able to obtain a lower final price for each of the units and the same will happen as well in a setting of a private sector. That will be the logic that is used to say why buyer power is a positive thing in both the public sector and both the private sector, so it is seen as efficiency enhancing because it allows to decrease the purchasing price of something without necessarily having an adverse impact when it comes to the amount of goods that you buy because when the price that you pay is being reduced by reducing the quantity that you buy, that seems a bit counter-intuitive, you enter into something that is called monopsony power and monopsony power is the negative aspect of buyer power because in monopsony power, you pay less for the goods that you buy because you buy less. It’s because the way that the supply curve exists, it’s that the more that you buy, the higher the price, so you can drop the price by purchasing less but when you purchase less, you generate inefficiency in the market because you’re not buying the optimal amount of goods that are required.

So this is what is portrayed as the negative side of buyer power in simple terms, it’s a bit more complicated in abstract, I think it’s difficult to grasp, but the problems exist in both markets. And let me give you another example, Pedro. For example, in the case of centralised purchasing in public procurement, the centralised purchasing is said to be positive because you manage to carry out a lot of different procedures into a single one. So you pull them together into, instead of 10 municipalities buying, each of them, their own goods, they just entrust that activity to the one central purchasing body that is going to carry out a single tender instead of 10 tenders at the same time. So not only do you obtain a lower purchasing price because you’re pooling the demand of all of this but at the same time, you’re reducing your administrative costs because you only have one tender to do instead of 10 separate tenders and that is seen as a positive thing because it reduces costs and that is something that the directive explicitly says, “Well, it is good because we’re going to obtain cheaper prices and also cheaper costs.” But at the same time, that can be also negative because what you’re doing there is creating a lot of concentration in the market where there’s only one buyer buying everything for all these other buyers so this buyer has a lot of market power and therefore makes it more difficult to, for example, I know it’s one of your topics of interest, to small, medium undertakings or enterprises to have access to those contracts because they probably don’t have the capacity or they cannot compete in pure terms of price with a larger seller.

So the contract can be very efficient in the short term because you obtain a lower, cheaper price but, in the long term, you can have a problem of market concentration because the smaller suppliers cannot compete any more. And this is something that also might happen in the case of private sector and in the private sector you see that in some areas there is not conclusive data but there is some claims that buyer power, for example, in the supermarket sector creates problems because the smaller suppliers, in this case, farmers, are being pushed out of the market and I guess that’s something that we all see in the newspapers, that there’s problems between the buyer power of retail stores and small suppliers that are being squeezed outside of the market and I think that same negative side exists, may also exist in the public procurement setting, although in the public procurement setting we have different ways of doing things, especially because we can, for example, divide lots, we can divide the contract into different lots. So if we do that, we kind of put away, aside this problem or at least make it less complicated or less difficult or less prone that concentration in the market arises. I don’t know if I am being too technical or…


No, no, it’s perfect. Actually, you have pre-empted my next question which is if centralisation can have that negative effect on the market, and that is certainly my view and my opinion as well, what about lots? Because lots have been bandied around as being a solution for all evils of centralised purchasing but personally I’m not entirely sure because, on the one hand, you’re effectively creating, again, the same transaction costs as you would if you had just different procedures. That’s the first one, and the second one is if the lots are not done well, you’re actually facilitating the collusion in the market.

 Yes and I fully agree with you and it’s a little bit of self-advertisement but last week a book came out in which I wrote about this, I wrote about what happens when you try to centralise and at the same time you try to divide into lots and those things don’t make any sense because either you do one thing or you do the other thing. And the problem with lot division is that lot division, I think, is actually a positive thing but firstly is not mandatory, member states can decide whether they want to let contracting authorities to do it or not while in some contracts I think lot division should be actually said, “In this case, you should, in 99% of the circumstances, divide into lots.” But at the same time, as you say, when you divide into lots, you end up in a problem that the cost saving that you managed to do when it comes to the tender, it’s gone or maybe it’s going to be reduced, because now you’re going to incur into a lot of expenses when it comes to contract administration because instead of having one single contract, you have maybe five or six depending on how many lots you have, and all this is going to be more contracts that you have to administer. So tender wise, it’s going to be cheaper if you centralise because you might say there’s only one procedure in which you have different lots but once you enter into the contract execution then you have the problem that you have to administer more contracts so that is also expensive for the contracting authority. The difficulty here lies in determining which of the effects is better? I mean, do we save more money by centralising the procurement and is it an efficient saving and we’re not hurting the market when we have a centralised purchasing body that carries out a lot of tenders and is this cheaper than administering five or six or eight lots? That’s one question and I think that we don’t have conclusive data on that area so it’s difficult to say which effect prevails.

And then you have the other problem that you mentioned, that is the issue of it’s not only about dividing into lots, it’s knowing how to divide into lots because if you divide into lots in a way that this just makes easy or facilitates collusion among suppliers, among economic operators, then the lot division is going to carry out a negative effect in the sense that for the contracting authority, it’s not going to get a cheaper price, it’s going to get a higher price because the tenders are entering into a bid-rigging agreement because they can’t seem to distribute the lots. So, for example, if you know that you’re going to have probably three economic operators submitting a tender, you should never divide the contract into three lots because then you know they’re going to split the market. So lot division has to be done in a smart way and that is a bit difficult to know because you have to have understanding of competition economics and you have to know how the market is going to react and that is a difficult exercise for a contracting authority.

So yes, I am more positive than you in a sense that I think lot division is positive if done correctly, the thing is that there’s very little indication how it’s to be done correctly according to the directive and that’s why I think that in these cases, when it comes to centralising and when it comes to lot division, what is really important is some kind of best practices. People in the central administration and people in the European Commission or different bodies in Europe should tell the contracting authorities, “We have these different tools to maximise public buyer power in an efficient way. We should be aware that you can exercise buyer power that is exploitative or exclusionary and is anti-competitive, that creates inefficiency in the market and you should use it in a positive way. And on top of that, you know that there is exists something called lot division and lot division is positive in certain times and you should consider using lot division in these kinds of ways.” So my suggestion would be we need to train people when it comes to how to carry out centralisation of purchases, what kind of things we’re going to buy and if we should use lots or not because I think lots are positive but, as you say, not always.


One of the problems I have with lots is that, as you said, it’s very hard to get well but it’s also very hard to forecast in advance, let’s say, a simple solution that would work in most cases and you’ve indicated one basic idea which is if you know more or less the number of tenders are going to come up, you should always have a number of lots that is smaller than the number of tenders.



But that’s pretty much and, from then onwards, it effectively implies that you know the market really well and it implies that you know or you’re going to be able to find a way to divide whatever you’re going to be doing into lots that actually increment competition and do not affect it. And personally I don’t see people in the field having that capacity to be able to do that kind of analysis.

No and I agree with you. I think for the contracting authority, it’s not because the administrative personnel or the civil servant is not smart enough to know this, it’s because he’s not trained to do it and I think that’s a problem because it’s not straightforward, you need to have knowledge on what you’re buying, you need to have knowledge on, at the same time, the market that is out there, that is going to supply you the goods, you need to understand how private operators operate which is very difficult because sometimes there’s a lot of information asymmetry that the one that buys doesn’t know that much of the market and the one that sells knows it much more. So it is very tricky and that’s why I think contracting authorities need some kind of counselling, they need some kind of training on how to carry out lot division in an efficient way because carrying out lot division for carrying out lot division is not a smart tactic, it’s just going to create an unnecessary amount of contracts and maybe you won’t even need to, it might not be a very efficient choice and then you might be sacrificing efficiency for the sake of having more players. 

But at the same time, having more players sometimes is justified and sometimes it is not, so it also very case sensitive and it’s difficult to give a straightforward answer. I think one of the bits, one of the challenges of public procurement is that it’s so case by case analysis because our markets are special because what we buy is different because the rules are complicated and because the needs of every contracting authority are as well different. So it’s difficult to give a very clear cut answer when it comes to this. I think the answer is training, I think the answer is telling the buyers out there, “What should we do about this? How should we do it?” And explain the pros and cons of centralisation and lot division that are two issues that seem to go in opposite directions even though the directive wants them both to come together in the same place. So I think that’s also a deficiency of the directive because the directive tries to do probably too many things that I think are well intended but when you try to put them into practice, they probably are not going to be able to be compatible with each other and that’s one of the problems that I see when it comes to the policy regarding centralisation of purchases but at the same time promoting small, medium undertakings. Either we do one or we do the other one or if we want to try to do both then we need to really know what we’re doing.


Well, about the directive, I mean it’s important for us to understand that it’s a text of compromise, it’s a compromise achieved between 28 member states and also the European Parliament. So it’s always going to be trying to achieve everything without actually hitting the sweet spot for anyone at any given time. And also the directive, it’s supposed to harmonise the legislation, it’s then up for the member states to transpose it into the national legislations and what we’ve seen is that, more often than not, the member states like to use the directive as an excuse for any mistakes and any failures of the system without them taking into consideration that their own responsibility into actually weaving the directive into national law. That is certainly the case here in the UK over the last 10 year that I’ve been here. Now moving on the ball a little bit further, I was listening to you and listening to your critique about centralised procurement and possibly the creation of a monopsony by the contracting authorities but the fact is, from a public sector perspective, the logic when we, or when they go for centralised purchasing, the logic that they want to achieve is actually the savings that can be the end result of that process. There’s no real consideration about the impact it’s going to have in the market because, effectively, they’re externalising the cost either to the market itself or either to the future and not to themselves today.

Yes, and one thing that I want to clarify is that I think monopsony power, both in the public sector and in the private sector is very rare and with monopsony power I have to be very specific with what I mean with it because the literature is a bit tricky when it comes to the use of terms, especially in the US. In the US they use monopsony power to refer to buyer power in general which I think it gives the wrong impression because monopsony power is one very specific model of purchasing that, to decrease the price that you pay, you buy less quantity. That’s one way of buying, there’s many ways of buying and I think in the public sector, exerting monopsony power is even more difficult because your demand in this condition, by the demand of the end user, that is the one that obtains the benefits of the public service that the contracting authority gives. At the same time, as you say, the public buyer has different interests than, for example, a private buyer and the public buyer has a different interest, it’s not profit maximising necessarily, what it wants is to obtain good quality products, what it wants is to satisfy a social need and might not necessarily enter into the situations in which it might abuse its power as a monopsonist would do.  

So I don’t think, in the public setting, I think monopsony power is not the main problem. What I think the main problem in the public setting would be the use of inefficient bargaining power in the sense that we obtain a discount but that discount either is being eroded by the cost of running the process or we don’t really use it or we don’t pass the benefit to the end consumer in a positive way. And this is one of the differences that I see between the public buyer power and the private buyer power because the public buyer has different interests than when it comes to a private buyer. The private buyer wants to maximise profit while the public buyer needs to satisfy social needs and that is a crucial difference between the two of them and I think that’s something we have to keep in mind. So I would say in a public sector setting, the case of monopsony power is going to be very rare, if likely impossible, and the same happens in the private sector. In the private sector, monopsony power, it’s no longer in fashion among the economists, now we speak about efficient bargaining and then we speak about different types of contracts that we can do, we can use to make as efficient as possible the contract and therefore reduce the prices without having a negative impact on welfare when it comes to study welfare.

So the ability to buy something without reducing purchases and not hurting the market as such. And, as you say, the public buyer is not aware or is not interested in hurting the market but it can do it as well and that’s why I think it’s important to understand that the public market should be disciplined in some way because, yes, public buyer power might be efficiency enhancing and might be positive if we use it right. And then we have two problems. We have problem number one, most purchasing by contracting authorities, it’s not covered by EU competition law rules, that will depend on whether the activity carried out with those purchases constitutes an economic activity and that’s the problem that Albert Sanchez-Graells and I, we have been working a little bit on this, Albert being the first one to put it out. And then, not only that we’re not, the public buyer’s not conditioned by competition law, which I think it should be, but the other problem is that sometimes we are a bit too naïve in the way that we buy and we don’t know exactly what is going to be the repercussion in the market of the way that we carry out the purchases and this is another factor that I think contracting authorities should be more aware of. That whatever the contracting authority carries out in the market, it can have a substantial impact, especially in sectors that are very dominated by public buying. For example in Norway we have the case of the health department, we have something that’s called Helsenorge and Helsenorge is the central buyer of absolutely everything that is carried out when it comes to healthcare in Norway and public healthcare in Norway is about 97% of the market. So whenever Helsenorge carries out a tender and buys goods, its activities as a buyer has a repercussion on the suppliers but, at the same time, on the end consumer as well because the end consumer is going to be either beneficiated or is going to be harmed depending on how the buyer behaves in the short, but more importantly, in the medium and long-run term. Because I think the key is using buyer power in a way that it is efficiency enhancing, it can reduce prices but at the same time, is used as a tool to maximise dynamic efficiency which basically means, in simpler terms, to make sure that we obtain the best outcome when it comes to innovation and equality as well in the long run.


So I’ve got two final questions for you before we finish and I’ll ask you to be brief on those. You see that intersection between law and economics, what methods are you using in your research, especially when you’re looking at centralised procurement and what challenges do you face? 

Well, the very short on this, I am not an economist myself so that’s the first challenge because we’re talking about very complex micro economics and industrial organisation. So what I’ve done is I’ve not used a full-fledged law and economic analysis which is what most economists tend to do when they do law but I’ve done something that I’ve labelled an economically-informed legal analysis which basically means I do a literature review, I try to understand what the economists are putting up, then I try to see if there is a consensus in the area and most of the time, there’s no consensus between the economists and then I try to say, “Well, this is the outcome and we don’t have a clear-cut answer because all the models by the economists are highly dependent on assumptions.” If assumption A does not exist then the model crumbles and falls and you don’t have the model working anymore and that is one of the main challenges. The main challenge, for me as a lawyer is that economists work with fiction and lawyers work with reality and I work in a centre that we mix competition lawyers and economists and when I speak with them, they tell me, “Well, this is my assumption,” and I tell them, “Well, but the world doesn’t work like that,” and they say yes, and they tell me, “Yes, I know it doesn’t work like that but that’s the only way that I can simplify the model so I can make assumptions and I can have reliable results.” So of course, I think economics is fundamental and very important but at the same time, we have to be aware that both our disciplines have limitations.


I mean, that’s a very important point because I’m always very well aware of the limitations of our discipline and how we usually are interested in, yes, analysing reality as a lawyer but we’re interested in a binary answer which is, “Yes, it’s legal,” or, “No, it’s not legal.” Whereas we usually do not look at the wider implications of any decision and that’s certainly the issue I’ve had with law in the long run which is if we just look at yes or no, legal or illegal kind of dichotomy, we are enabled to actually help whatever area of the law we’re studying, actually developing into achieving whatever goals the law is supposed to achieve.

Yes, I agree with you. I think we have to skip out of a yes or no answer because a lot of our answers, as a lawyer, are in a grey area. So whilst I say all the time, the answer is, “It depends.” It is very difficult for me to, especially in the field of anti-trust to say this is right or wrong. Well, it depends, it depends on what is the situation, it depends on the variables at hand, it depends on, as you said, what are the goals that you’re trying to give? Because this is another point, what are we looking for? And depending on what are we looking for, our answer’s going to be tailored in a different way because we try to adjust the way that we interpret the law to fit our result that we desire. So it has to do a lot with also the political goals or the social goals that the rules try to achieve and that’s how we also use our interpretation or our way of applying the law to achieve those end results which is also a highly sensitive topic. And the economists also do the same even though they say they don’t but I think economics is also value-laden in the sense that economics are based on a specific set of beliefs so, for example, if you believe the market is the right thing to do and that will be the Chicago School of Economics, everything is about end consumer efficiency. Or you maybe believe that the market should not be left alone, there has to be some kind of regulation on it and then you have to have some strong rules that are going to tell you how to behave and that would be maybe the Ordoliberal school which is a school that I am more interested on or maybe you say, “Well, you know what? The market doesn’t work at all and we have to have a state plan control,” and then you end up in the situation like as a socialist or a communist state.


Like Venezuela? 

Or Venezuela in which we don’t know what’s going but something is going on and there’s probably more like a capitalism of state which is a different thing.


OK, one final question which has to be really, really brief because we’ve gone over the allotted time that we had for the interview. What future work are you planning in this area?

Well, in the area, I’m going to be working mostly now, my postural project on utilities and regulation of electricity and gas. So I’ll be looking into the utilities regulation of gas and electricity and particularly to see what happens when the activities are exposed to sufficient competition, that they get a waiver from the application of utilities directive and see how this waiver process is being carried out and how the practice is done because what I’ve seen is that we have very little information on it. And then I will continue working on purely competition law, anti-trust law, also in the field of electricity and gas when it comes to access to distribution networks and non-discrimination for third-party access.


Looks fascinating. Ignacio, thank you very much for taking the time to be with us today.

No, thank you, Pedro, and I’m sorry for speaking way too much, that’s what happens when you get a Latin American in your show. I mean, we speak way too much but it has been a real pleasure for me, thank you very much.

#26 - S. N. Nyeck (Canterbury Christ Church University)

Public procurement governance in Africa

Interview with S N Nyeck from Canterbury Christchurch University on public procurement governance in Africa. She is generally interested in the political economy of development and the role that public procurement plays in transforming institutions and societies and recently edited a book entitled Public Procurement Reform and Governance in Africa, published earlier in 2016 by Palgrave Macmillan.



I found your book when I was looking for new publications in public procurement and I really like that we finally have a very good volume, it’s not the first one but it’s a very good volume on public procurement in Africa. So why did you decide to edit this book and what you were trying to achieve with it?

Well the idea of this book emerged when I attended a conference in South Africa on public procurement. What struck me at the conference was that most people attending were either lawyers or practitioners to some extent of public procurement in different institutions. My background is political science but I also have some background in law. What struck me during the conversations (pause) most presentations were really about either the legal aspects or institutional aspects of public procurement reform or some of the challenges that are faced when it actually comes to implementing a reform, but by and large the conversation was very much heavily framed by legal questions and every now and then at the end of every presentation I will hear someone say “well, well done, fine, but it doesn’t work like that in practice”. Right, it doesn’t work like that in practice, it is something that I kept hearing over and over again. And I was sitting there and thinking this is really a problem of not having the many fields that are interested in public procurement reform talking to each other, right. Legal scholars don’t necessarily engage with policy or political scientists, you have people in the business side not necessarily interacting with either legal scholars and/or political scientists. So I saw the necessity really and the urgency to talk or at least to propose a framework that would create enough synergy and allow practitioners and researchers to deal with a question of governance and public procurement in the developing and middle size countries, developing world or global south, in an efficient way.

The issue here is that we are confronted with several questions. One is the issue of institutional building. In many countries this is an ongoing process, unlike developed countries. Second, we are confronted with the changes that are outcomes of reform and we are concerned also about the economic performance of reform. So in light of these concerns, in light of those debates at that conference, in light of what I know about the significant place that policy and politics really plays in any reform, in any public reform, I wanted to do something about the subject and in fact the subtitle was ‘an exploration of the law, politics, business matrix’, okay, that doesn’t necessarily appear in the final cover but that was really the intention, that was the original inception of the ideas and that is what motivated me to do or to present scholarship that is really intersectional, that shows the limitation of a one-size-fits-all approach to reform and governance of public procurement in Africa.


So the starting point for the discussion is effectively your view that a few years ago when that conference happened, which was maybe 2010, 2012?

Yeah, around that time.


That public procurement reform and governance in Africa was being analysed only from one specific perspective or one specific discipline?

Right. I think that it is not just my opinion, it is a matter of fact, okay. The impetus for reform in Africa have been concerned about global trade, okay, the idea that free trade is likely to bring more opportunities to people and government everywhere, the idea that competition is likely to bring efficiency, and the idea, really mostly arising from the experiences with government over the past twenty years or so, that there is a lot of inefficiency in government and that sometimes there might be creative ways to fulfil public missions for the benefit of the constituencies and citizens. So it is that international context that informs public procurement reform in Africa that is really, I haven’t seen one single case of substantive consultation with society, with stakeholders prior to a reform anywhere in Africa. I think it is a fair statement to say that it has been by and large a top-down model facilitated by institutions such as the World Bank to some extent by some professional organisations, but by and large this is something that in fact most people are not even aware of, you know, on average people don’t know what public procurement is about.

And I tell you what, I was speaking to a group of African scholars here in the United States and I’m talking about public procurement and the importance that it has in understanding development today and in shaping really development, and as I was talking people started leaving the room one by one. No-one had a clue what I was talking about, you know, and it seemed so obscure. Now that is not necessarily the state of knowledge in Africa, it is a global trend, but what I’m trying to say is that by and large, and one measurement really for this will be to look at what is civil society doing, in which places we see an active civil society engage in say monitoring procurement, in providing feedback, that is, you have to look really hard. Nigeria in fact is the only African country with legislation that provides for civil society and professional organisation to actually be part of public procurement processes, and again that law is on the book, doesn’t mean it works effectively. But the point here is that if we were to take governance then, right, as this multi-stakeholder approach to development, it is really striking to realise that after I would say ten/fifteen years we do not have civil society involved in this. So then the question that we can answer is who else is involved? Okay, who is driving reform? It has been done by as I said different impetus. One is to bring this idea of competition which is great, the other was a concern about bilateral and multilateral trade with the EU but also China to some extent. So those sort of sources of reform are great but what I’m trying to say is that we still have a lot to do in Africa.

The last point I’ll say a measurement really of how much governance is governance in public procurement is that as the recent book by De Mariz, Menard and Abeille shows is that the officers, the public procurement officers were not involved and were not thought of as a major constituency in designing public procurement reform, and that is a statement from a former World Bank director responsible for operations in Africa, right, what was thought of as a priority was really amending laws, changing laws, but not really giving a lot of thought in terms of capability. How do we actually engage? How do we train? How do we put resources so that we are renewing the public sector, we’re training more people, we’re encouraging civil society to be part of this, we are monitoring what is happening? And so if you take those two constituencies, the public servants themselves or public procurement officers and then civil society in general, you see almost nothing. A very weak patchwork of attempts to actually practice governance in the public procurement sector in Africa.


I was listening to you and it reminded me of an interview that I did recently with Ana Cristina Calderon Ramirez, which I’ve just published on the podcast, and she did a lot of research into South America and how actually it wasn’t the changes in the law that led to improvement in public procurement practice, for example in reducing corruption, but it was further reforms that were done afterwards, for example the creation of regulatory agencies that had higher impacts than the laws themselves. And if you think about it, okay so it may well be that Africa is just a little bit further behind the curve than South America and the Caribbean in that sense that we start with the laws and that’s where the emphasis is happen in the beginning, and then you move on to the other elements of the public procurement stack so to speak and the next one might be the creation of regulatory agencies, might be the training of officials, of improving procurement practice. But your point is well taken is that at least for a good while the emphasis is pretty much only on the legal aspects of procurement and not the practical ones?

Right. And I was saying maybe Africa is behind, I don’t think it has to be behind, right, and I think that one of the excitement that I have in studying public procurement, right, is that unlike the traditional way of understanding development challenges, public procurement really allows us to start thinking in terms of similarities as opposed to differences, right. So yes, the traditional way would be to look at what X is doing and Y is behind because Y will catch up in due time, but when I look at the United States, state level issues, I mean here in New York we just had a commission on governance in public procurement and what it was revealing were stories that are very similar to what I read in terms of corruption, in terms of networks of waste of public money. In Montreal the same, that is another commission there, the Charbonneau Commission unpacking criminal networks in the construction industry, and when you look at what is actually happening on the ground there is a lot more that we can learn from each other and this idea of perhaps one person starting and the other one catching up I think for me it is a little bit depassé, right, because the urgency, the impact is not going to wait, right, we’re not going to wait for Latin America or for Asia to do it. We say that this reform is happening and there are ways that we can contribute to inform practices and that Africa does not always have to be taking practices or theories from elsewhere. And this idea of thinking procurement as a matrix, as a law, politics, business matrix, developed in the book, is really about putting the continent at the forefront of the debate, actively contributing and giving feedback to reform and pointing to things that are not working and pointing to others that are actually working and efficient. So yes, I have no doubt that maybe in some cases some models can be replicated in Africa but in a lot of other cases I think that Africa has contributed to this debate and can contribute to this debate on its own terms.


Speaking of the book, what were the major contributions and disciplines involved in it?

Yes. I made a decision to really have an interdisciplinary approach to this book. And one key distinction from the previous books, two that I know of on public procurement in Africa, is that political scientist, policymakers, legal scholars, public procurement practitioners, but also really civil society contributes to the debate here. And I think that the fact that we rarely hear from practitioners and that we haven’t started really paying attention to civil society is something that this book sticks to correct. So disciplines as diverse as they can be, myself as I say I’m a political scientist trained in the political economy of development, but you will find professors of accounting and strategic management, you will find legal scholars, people coming from quantities surveying, sciences, people working at the UN and governance, and from countries as diverse as Ethiopia, South Africa, Kenya, Uganda, Botswana etc.


There is a quote in the book that I quite like from your introduction and it reads “it also however acknowledges the symbiotic relation between formal institutions, society, culture and politics, as important variables that add to our understanding of what works and what does not yet in public procurement”. This is a quote from your introduction. So in the end what goes into procurement in your view?

What goes into procurement is not just finite transactions, okay. So a transactional approach, institutional approach or economic approach to procurement will tend to maybe emphasise good works at finite deliveries or needs or supplies that go through a procurement. But values go into procurement, right, and power goes into procurement. These are tangible and important assets that we should also be paying attention to.

Let me maybe illustrate what I’m trying to say here with an example coming from South Sudan. I had a privilege to be in the country meeting with governors in 2013 prior to the sacking of the vice president leading to the current war. And I attended a meeting where governors from different provinces were talking about, you know, what is working and what is not, and this particular governor told a story of a contractor who secured a contract from Juba and it was about land and exploiting land for commercial purposes, except he never went to the place himself and he never interacted with the populations there. So he arrives with a contract, legal contract, legal transaction, just to find that the actual land is a cemetery that has been there for a long time, and he comes in with his bulldozers, you know, trying to destroy everything because “hey I have a legal document, right, and I’m supposed to be doing this”. So right there the issue is not about the legality of the transaction, there is something else going on there and we have not paid attention to these sort of important issues.

The other example that is raised, and this is coming from the book, there is a chapter there really raising the question why African government sign bad contracts, right. And sometimes we are caught up in the legality as manifest in law and written documents but this chapter does, is to really argue that the pre-award period is just as important as the award and the post-award period, right, and the contributors really show that a lot of corruption happened during the pre-award period. And therefore it’s not necessarily about legal or not legal, a different question of just human interaction, right, how they happen, what goes into it. And by the way these are not questions that are limited again to Africa, here in the United States we’re talking about the influence of money in elections and kickbacks tied to government contracts here and there, right, so we’re talking about how things such as the political process, how things such as democracy can be indirectly impacted by what happens in public procurement regardless of its legality. Other works coming from Europe, actually there is a fantastic book by Anna Maria LaChimiatalking about tide aid and development procurement in the framework of the EU and WTO laws, so this idea of tide aid, it is really about something else. It’s about value creation, it’s not just about what goes into the transaction, right, it’s about conditionality and the fact that terms of agreement are not necessarily working or supposed to be working for some parties.

So what goes into this from all of these examples is that a finite’s good but they are also about political goods, they are about economic goods, and this relationship should be thought of really carefully especially when we’re talking about development, in the context of development. So formality is great, formal institutions are absolutely needed but society matters, okay, culture matters, politics matter, human rights matter. And that is these are substantive questions of equality, of quality of society when the government is no longer the source or the sole source that provides the works, good and services that we need as political entities.


And you think that those problems and issues that you’ve highlighted and mentioned are they transversal to the continent or they are more prevalent in certain countries in comparison with others?

The issues that I’m mentioning here are global issues, and as I previously said public procurement is really, allows us to look at, or at least encourages us to approach comparative analysis in different ways. So the question of civil society involvement and its capability of involvement is one that is still very much part of a conversation in the United States, right. The question of the future of democracy, given government outsourcing, is a global conversation. Yes, certainly countries are different but I do not think that these problems are Nigerian problems or Senegalese problems or South Africans problems. And because in part the impetus to reform public procurement is a global one, a lot of what is happening is really happening at the global level and then influencing the ways in which different countries choose to be part of the conversation or not. The question of gender equality for example in public procurement is one that I see coming out of the works in places such as England, right, and that is an issue on the continent. It is not a Senegalese issue, it is not a Cameroonian issue, right. So here I want to say that the substantive when we move from the traditional frames of comparison, is it a French colony, ex-colony, is it a British ex-colony or Portuguese ex-colony? To look at laws as part of a global trend, a global movement, a global movement that harmonises practice and trade practices, and then looking at what are the substantive impact of these laws? What are the substantive impact on institutions? What are the substantive impact on values and society? These become issues that make us look alike. And that is my take on understanding, studying public procurement reforms. They are global problems, they are also in Africa as elsewhere.


One final question. What is next for you? What are your future steps in research terms?

I would like to in my future work tackle a little bit more substantive questions such as human rights in public procurement, gender equality in public procurement and there is not much done also in that area. So I am interested in also bringing the experience and conversations from Francophone countries to the debate. So in terms of my perhaps regional focus I would like to do a little bit more work in Francophone Africa. In terms of my substantive engagement I would like to take government, or engage governance for what it is looking at the economic impacts of reform on the quality of life, the impact on relationships, you know, gender equality is one thing that I care about, but also impact on the environment. So these are generally speaking areas that I’ll be happy to explore within the next couple of years or so.


I think that’s a very interesting research plan that you have and one that I would like to know more as you go on. Thank you very much for coming.

Thank you very much for having me.

#25 - Ana Cristina Calderon Ramirez (Maastricht Graduate School of Governance)

Regulating public procurement in Latin America and the Caribbean


Interview with Ana Cristina Calderon Ramirez from the Maastricht Graduate School of Governance on the regulation of public procurement in Latin America and the Caribbean. Ana is currently studying in-depth the creation and establishment of public procurement regulatory and monitoring entities in Latin America. She co-authored a paper entitled ‘Elements of public procurement reform and their effect on the public sector in Latin America and the Caribbean’, which was selected for a Best Paper Award in the International Public Procurement Conference 7 held in Bali in August 2016.


Hello Ana, welcome to the programme.

Hello. Thank you so much for inviting me.


Let's start by talking about your research in the paper that you’ve published with your colleagues. What is the background behind that paper, so what led you to actually write it?

Well I must say first that the story of the paper started with the research in which I have been working as part of the PhD studies at the School of Governance in Maastricht University as you mentioned before. When I was starting to explore the topics that I wanted to study, it appeared very interesting to analyse the elements of the procurement reform in Latin America and the Caribbean region, including the improvements of regulatory frameworks, the introduction of electronic government procurement tools, and the creation of specialised regulatory and monitoring procurement entities. In doing that I was looking for data, quantitative data to know and to understand a little bit more  about the historic  patterns of the reform and when that happened one of my colleagues and friends, Jorge Muñoz, who is the other author of the paper, and that happens to be econometrist specialised in public sector, told me about the PRODEV evaluation dataset that is compiled by the Inter-American Development Bank and that compiles the indicators related to the advances in public management in the region. So we started with him to check if there were any patterns or relationships in the reforms that have been happening and improvements with other indicators or variables including public expenditure or in institutional strengthening.

A year ago we found out that the creation of those procurement agencies was very relevant during the period of 2007 and 2013. Then we were talking with the other author, Leslie Harper, who is a well-recognised Fiscal Specialist at the bank, and with Jorge and Leslie we started to have conversations after work thinking what we can do with these indicators to meet the demands of our client countries. We have been asked for more information on these issues and we wanted to fill a  gap that is in the literature. So we decided to write a paper for the Procurement Conference number 7, , looking for the relationships or effects that the improvements of public procurement system have in the  perception of in our region. So I should say that this is totally a team effort that combined policy and research.


Sure. It makes perfect sense. But you got the idea to do that paper based on your work at the bank or based on your research, academic research that you’re doing for your PhD?

Well it’s a mix, you know, because when I started the PhD as I mentioned it I started to look for these datasets, we found some interesting things and then talking to Leslie who was my supervisor and Jorge,  I said “well we have so many clients needing information, policy information, we should do something with this” and she said “great”. So it’s when you find two ways that they meet together, you know, policy and research, so it was the two things.


Talking about the research that you’ve done in a little bit more detail. Why is it important and what aims were you trying to find out once you started working on it?

Pedro I should say that from an academic perspective, as we know much of the research in this field concerns itself with description, others with explanation of public procurement reform process. In Latin American, Caribbean region more than twenty years have passed since the reform began and since then there have been great strides in the modernisation and reform of the public procurement systems. These advances have included, as I mentioned before the creation of e-procurement platforms, the establishment of more comprehensive legal frameworks, the professionalisation of procurement officers, the use of procurement to promote social objectives, and inclusion of the green criteria in tendering opportunities.

Public procurement reform in the region has been analysed in issues of the impetus of the reform, the issues and constraints affecting these reforms, partial evaluations on specific country cases. However, if you look in the literature little has been written to identify and analyse the outcomes of these reforms and the impact that these reforms have had on the perception of public sector performance. So with this paper we wanted to address those gaps in knowledge by providing a conceptual framework for understanding the context under which public procurement reforms have been carried out and examining what we have called the first generation public procurement reform elements and the impact of those elements on public sector performance.


Those are the improvement in regulatory systems, electronic procurement and monitoring entities?

The three of them. The creation of electronic, yeah, the law and the institutional aspect, yes. Going back to the point that you mentioned before about the policy, I think we achieved four main things that are good for our colleagues in the international development field because first we were able to provide a framework that connects public procurement within public sector literature, policy literature, although these things very obvious in the public procurement has been an isolated topic, a ghetto topic. Sometimes we know that it is part of public sector administration but there is a lack of understanding of its importance and the role that it plays. We have seen public procurement reforms as part of huge reforms, public financial management reforms or anticorruption and transparency reforms, but not as a field that has its own space. So now it’s more like it has its own agenda, so we add that into the map of public sector reform and the elements that are included in that.

Second, we made an effort to clarify the effects of these reforms, both in Latin America and the Caribbean because most of the literature and policy literature covers Latin America but not the Caribbean. So here we use the dataset, the complete dataset with Caribbean countries and this is very interesting. And second, we tried to make clear what was first and second generation reforms and effects of these issues in other important sectors. Third, we demonstrated that although there is not much public procurement data as there is in other subjects, we can use real data, especially that information that has been collected by agencies, international development organisations and the countries itself. So sure we need more data and this has been an issue for so many years but now we have data and we have to work with that data so let’s start to use what we have.

And finally, fourth there was something very important to us, this is a research with a policy perspective as well. Because sometimes you heard that research is research, full stop. If it’s used that’s another stuff, however we tried to combine policy and the academic world in this paper. For the people who are listening to this podcast I want to let you know that we have a big network on government procurement in the region that is called the INGP and we have so many conference as part of these network and so many trainings, and there we have heard so many people telling us that the research is very hard to understand and other papers are very light. So we wanted to combine that both academics and policymakers could enjoy and use what we wrote and more important what we found.


I think that’s a very important skill. One of the things I don’t like in academia in general, and I used to be a lawyer before I became an academic, is that so many of our colleagues write for themselves and their colleagues as well and they don’t understand or they don’t make an effort to actually write aiming to generate impact or at least so that their research can be understood and digested outside academia.

That’s great that you say that because that’s what we are doing now and that’s what I’m doing as part of this PhD, that is very different programme because it’s for policy people, so that helps a lot.


So you’re looking at some indicators and the effects and the impact that certain reforms in public procurement had had in the Latin America and also the Caribbean. What were your main findings?

I have here to make a clear point. These findings are for a dataset that was between 2007 and 2013, so let’s make that clear first. On that regard, based on the three main areas of the procurement reform evaluated we find out that the creation of a procurement agency had the largest impact on the development of public procurement systems between those years. Indeed the increase of the scoring of public procurement system in the countries where a public procurement agency was created was about 0.5 points above the scoring of the public procurement system in those countries where an agency was not created.

Second, there is a link between the creation of a public procurement agency and other variables related to the perception of public sector performance and the competiveness of the countries as well.


So bearing in mind that you didn’t analyse in detail the characteristics of each public procurement agency because they may be different in different countries, so that is just a general…?


Yes. But as a part of the dataset you have that, you have the different characteristics so some of them are not agency, some of them are just entities or  a department within a ministry. So the dataset includes that.

The effect of creating a new procurement agency were also statistically significant to explain improvements in variables such as favouritism in decisions of government officials and transparency of government policymaking, the government procurement of advanced technological products and their competitive index among others. For example, with the regard to the perception of favouritism in the decisions of government officials, the creation of an agency may have helped to improve this indicator by 2.6% as a consequence of its monitoring role, ensuring that procurement processes are carried out in accordance with the country’s legal and regulatory framework in a fair and impartial manner. Additionally, by lowering the cost for entry, the procurement processes become more competitive and less likely to be perceived as biased towards a particular individual or firm. And the creation of an agency had a gross impact of 9.9% on improving transparency of government policymaking. This effect might be explained by the development and monitoring of technological tools, electronic platforms for example, when public procurement agencies are fully functional and compliant with transparency regulation.

Second, there is a link between the creation of a public procurement agency and other variables relating to the perception of public sector performance, as you can see in the paper, and the competitiveness of the countries as well. We found out that the effects of creating a new procurement agency were also statistically significant to explain improvements in variables such as favouritism in the decisions of government officials, transparency of government policymaking, the government procurement of advanced technological products and the competitive index among others. For example, with regard to the perception of favouritism in the decisions of government officials, the creation of an agency may have helped to improve this indicator by 2.6% as a consequence of its monitoring role, ensuring that procurement processes are carried out in accordance with the country’s legal and regulatory framework in a fair and impartial manner. In addition by lowering the cost for entry, the procurement processes become more competitive and less likely to be perceived as biased towards a particular individual or firm. Additionally the creation of an agency had a gross impact on, let me read this, of 9.9% on improving transparency of government policymaking. This effect might be explained by the development and monitoring of technological tools such as the e-procurement platforms or e-GP systems, when public procurement agencies are fully functional and compliant with transparency regulations. Also the creation of an agency shows an impact on the burden of regulation as it brings more transparency and competition when a procurement process is planned. Because of its strategic vision and understanding of the issues that are happening in the country and the needs that the country has, the procurement regulatory agency is able to develop a clear and appropriate regulation increasing private sector trust in public procurement and generating increased participation in the public procurement marketplace.

And third, the most important part I guess is that having a public procurement agency that is fully functional is absolutely critical for the reform process itself, at least in our region that has been a very important issue because an agency that is responsible for policy and monitoring is needed to balance the competing goals that the country has, whether it’s fighting corruption, promoting transparency or trying to achieve value for money effectiveness and efficiency. An office with just administrative functions and no strategic mandate, focusing only on transactions, is not going to be able or to have the incentives to advocate for deep and sustained reforms. In our region legislative and technological reforms have been very, very important but without an agency that can provide a strategic direction, leadership ambition to leverage those changes in a holistic fashion, the reforms are going to stall or be implemented in a disjointed and piecemeal way.


Can you give us examples of those agencies?

Yes, sure. For example  Guyana. Guyana has a great law and you can call Caroline Nicholas about this to ask her. She has been there for a while and she has help us in the region for our regulatory frameworks, you cannot imagine how much. But they had a beautiful law as we said in Spanish, a beautiful law, but the implementation has been very, very hard. They started last year to have a procurement entity to be responsible of the implementation of the law, bringing more transparency by having and monitoring the e-procurement system and this has been helping a lot. The same in Colombia. Colombia’s a great example and now Colombia is very fashionable because, we are trying to modernise the public sector. The agency is new, it was created in 2011 but it started in 2012, and these years have been very important for the country in the public procurement system because this agency has given to the public procurement system impetus, you know what I mean.


Yes, impetus.

Now people recognise this public procurement, they understand, they know that they can take and bring to the public marketplace, they can offer, they can buy, it’s easy to understand. So that helps a lot. The agency helps a lot.


Very well. Still on the main findings part of our conversation, I found interesting in your paper that you think the regulatory regime and the legal framework is not as important to improve practice or to achieve the outcomes that it’s supposed to achieve than in comparison with the procurement agencies?

That’s the reason why when I started to talk and said “please we should be very careful on this, this data is between 2007 and 2013”. Before 2007 I am sure if we go before that we could find that the legislative and the legal framework were very important, were the most important bit among those other factors. The thing is that in 2007 most of the countries already had the legal framework established but not implemented.


Okay, that makes sense.

So 2007 and 2013 was like an explosion of these entities and I explained in more depth in my dissertation that I call “agencification” . I think we should make this point clear.


So in fact you’re saying is that without the law and the regulatory framework already existing it would be hard for the procurement agencies to actually achieve anything?

Of course. Of course. And more in our region that’s very important. No, no, no, we should say that.  I mean, you cannot stay forever with a perfect regulatory system but no-one complies with that regulatory framework, you know?


Yes I know. I know the feeling. Very well, okay. So those are the main findings of your paper. You’ve been very cautious saying that okay this is only, you only analysed the data between 2007 and 2013 but can we extract wider application of the main findings? Do you think that they would be valid or would be useful for other areas of the world like Africa or East Asia?

Well I think with a good dataset we could do something with Africa and Asia. And as part of my dissertation, not as part of this paper, I have been having some interviews with a specialist at the African Development Bank and the Asian Development Bank and World Bank. But it seems that they are very different in the form that they are building up their systems, when I say system there’s no electronic system but the complete public procurement, system. For example, isomorphic pressures are different, the way of the path dependency is very different in those regions of the world. However, if we can find good data maybe we can find the same things, agencies will help to boost the efforts of the reform, but I wanted to be very careful on that point.


Your paperwas selected as one of the best papers in the International Public Procurement Conference that happened in Bali and that’s, it’s a great achievement for you to have been able to do that as a PhD student. What else are you doing now at this moment in time in term of research? So you looked at Latin America, what else are you doing for your PhD?

Well I’m not doing anything regarding the Caribbean right now. Maybe I will write a chapter for a book that is going to happen in 2017 with Inter-American Development Bank . Nowadays I’m just focusing on the public procurement agencies, not just the departments or small units but the agencies in Latin America. But I wanted to check the isomorphic pressures and the policy transfer pressures in the process when you establish these type of agencies. So that’s what I have been doing right now. However, I think I want to mention something if you don’t mind?



Because we have been doing some other stuff and I think the wider applications of these main findings could be from different sides. So from the country side I’m sure when the conference of the regional network and public procurement that is going to happen in November in Jamaica I’m sure the countries will talk about these issues and I am really sure that those countries that do not have these type of entities can learn a lot on their benefits and their impact, not just for the procurement system but for the public sector as well so they can sell more easily these type of reforms to their ministers. And others that have the agency on paper will understand the importance of the implementation because sometimes they have it there in the law but they haven’t had the time, the money or the willingness to make it real. For the donors it shows how the reform has succeeded in the region,  how much of the money that have been provided for the strengthening of procurement system is having a real effect. Actually in the paper you will see that we found how was improvement related to each dollar that was provided by the Inter-American Development Bank, so that was a good one for us. And it provides us an snapshot on what is happening in the sector, so that’s good for them to know how and where we are right now.  So I think that’s the wider application of the paper.


One final quick question to wrap up the interview. What future steps are you going to take in terms of research?

First keep collating data. Yes, because we need to collect data and we are doing this in the countries, donors. Maybe you can have another podcast on the methodology for the assessment of public procurement system that we have been working with the OECD. They have it right now for discussions in its website.


So they are reviewing it at the moment?

Yeah. They’re reviewing it. We were part of a taskforce that reviewed the tool, so that would be very important to us because for developing countries we use that to monitor the reform. The country data as well, the initiative of open contracting and, or open data have been helping a lot in this task, so we will keep collecting data.

Second, more research, more focused research, maybe about each innovation or change, especially e-GPP or green procurement, new entities or so forth. I know right now and. We should have more research on the quantitative impact of the reforms because mostly the literature that we have is anecdotal. Probably the analysis of the second generation reforms in public procurement, green criteria in tendering.

Third future step is to complete my dissertation work, crossing my fingers , I think it’s time.


That should be the first one I would say!

Oh my god! But here I wanted to talk about the paper I mean I was trying to be fair and say “well we’ll be collecting data, we will do more research” but yeah, you are totally right for me, it’s completing my dissertation. And  when I finish I will let you know about that for sure.


I think that’s a great way to finish the podcast. Ana thank you very much for coming.

Oh thank you so much for this. I really appreciate it that you inviting us.

#24 - Warren Smith (Digital Marketplace, Government Digital Service)

The UK's Digital Marketplace


Interview with Warren Smith, Director, Digital Marketplace, part of the Government Technology in the Government Digital Service. He enthusiastically and wilfully oversteps the mark to meet user needs in public procurement, in public sector procurement and contracting. He’s currently transforming the way that the public sector commissions products and services by thinking small about big problems, and thinking big about small successes.


It’s good to have you here, I’ve had you in the list for an interview for some time. Every now and again I like to interview someone from practice, who can bring a completely different view and perspective to what we would call legal, or academic problems, related to public procurement. So it’s great to have someone with a wealth of experience working in practice with public procurement. Speaking of that, could you tell us a little bit about your background, and what is a Digital Marketplace?

Sure. So I have been in procurement for 20 years now, I think. When I was fresh out of university I went straight into a procurement role. Really it’s only in the last five years, I would say, that I’ve been involved in quite transformative projects: working at scale and having a real opportunity to make a significant difference by introducing new ways of thinking about procurement and contracting.

I’ve really been focusing on trying to introduce the concept of user-centred design into procurement and contracting, as well as focusing more on outcomes rather than detailed input requirements, as well as introducing open approaches. 

What does that mean? It’s kind of being much more open about thinking, sharing information and visions, way in advance of actually procuring: so trying to engage with the market earlier. Also open standards and open data have a very important role. For me, making things open is a really fundamental tool for disruption, I think, which we can employ in many different areas. So that’s a little bit about me. 

I am a member of the Chartered Institute of Procurement and Supply, but I tend not to, to shout about that, but it’s a useful background, obviously, in the space that I’m in. But really my interest is in trying to take the focus away from procurement being a driving force, and make it, rather that procurement and contracting are enablers of better public service delivery. That’s a really important point. Often people who I’ve spoken to see procurement as being the end in itself, rather than a means to an end. That’s what’s brought me into the government. 

I had the good fortune of being asked to help GDS towards the end of 2012. I was brought in by the former programme director for the transformation programme, which was focusing on delivering the 25 exemplar services. These were the 25 largest government services by transaction volume, and it was the start of, really, the digital by default agenda within government. 

At this time different policy contexts were being set, Liam Maxwell, the former Chief Technology Officer had been brought into GDS to bring about reforms in the way that the government thinks about, mainly digital and technology. My role was to help rethink procurement in order to support those reforms of digital technology in government. 

So I’ve been here since 2012, and just something to correct you on, Pedro, and this is hot off the press, this week I’ve been appointed as the Digital Marketplace Director. Having been through an interview process, I’m now no longer the interim director, I am the appointed Digital Marketplace Director. 


Thank you very much…

Speaking of the Digital Marketplace, what are its objectives, and what are you trying to achieve with it?

I think you summed it up quite nicely in the introduction, but basically we are all about helping the public sector buy what it needs to deliver great digital services.

This is a key strategic component of the technology group within Government Digital Service, and it’s important that I just go back to that mantra there of helping the public sector buy what it needs to deliver great digital services, because that, again, is the outcome that we are trying to support. 

Everything we are trying to do is about transforming procurement so that it can help that aim of better public services that are digital by default. We are looking at the entire, end-to-end process of procurement and contracting, and focus on the areas where we think we can make the biggest difference, deliberately, taking thin slices out of that end to end process, applying user-centred design to that, and then iterating wildly, so that we can continue to deliver better procurement and contracting experience for buyers and suppliers. 

Those are our real primary user groups: buyers and suppliers. At the moment we are looking at frameworks quite specifically. I know you, yourself, have spoken about that in some of your publications, Pedro, and I think there’s a real interesting opportunity here. Within the digital and technology space in government we are looking to disaggregate large requirement sets into capabilities that meet user needs, and thinking more about how we might diversify and open up supply chains into government. Actually from that what you can then start thinking, is that the work packages and the contracts change into shorter term, lower value and quicker delivery [pieces of work]. So therefore, the sub OJEU threshold might actually be a really interesting, untapped opportunity there. 

We’re developing the Digital Marketplace so that we can award a framework agreement, then provide an end-to-end buying process where buyers and suppliers use the framework, get through the evaluation process, award a contract, get on and do the delivery, using things like the Open Contracting Data Standard to improve the quality of the data that’s actually the published automatically out into wherever, contracts finder or similar.

Can you give us examples of changes and simplifications that you’ve done to the procedures and contracts for the digital services framework?

Well, actually, we started on that with the G-Cloud framework, Pedro. G-Cloud 7, as in the seventh iteration of the G-Cloud framework, was the first time where we ran the entire OJEU  procurement through the Digital Marketplace, so that enabled us to really challenge the  -- I have to be careful of my use of the word “challenge” describing procurement -- but challenge CCS and government legal department to think differently around  what are traditionally the selection and the award questionnaires that you would normally see. 

We looked closely at the invitation to tender templates as well as the framework and call off contract itself. That was where we changed the supplier application process. A supplier simply had to complete what was called a supplier declaration, where they answer a series of questions which typically would have been the selection questions. That became a page, or several pages within the Digital Marketplace where we reused the design patterns for form elements to ask the right questions in a different order than normal. 

What we wanted to do was ensure that if a supplier was new to government, or even any supplier, it was easy for them to quickly decide, “Is this an opportunity that I want to go for?”. 

We wanted them to be able to make that decision as early as possible in the application process, for them to be able to decide, “Actually, no, this isn’t for me, I’m going to decline,” rather than going through a load of questions, only to realise that actually, “This isn’t an opportunity that either I am interested in, or isn’t relevant to me.”

It follows the principle of fail early, fail fast. We found that what might seem like a logical order of questions to a procurement person or a legal person, wasn’t a logical order to a supplier, and we’ve found that out through quite simply asking the suppliers, and undertaking user research with the suppliers to understand, well, ‘what are your needs, and how can we design an application process to better meet those needs’? 

That was really a fundamental point in time where we changed a supplier’s experience of applying to be on a government framework. That’s really the first moment where a supplier who maybe has not previously been interested in doing business with government, that’s the first moment where we’re saying, “Hey, come and get involved with  government,” and so we think it’s really important that that first experience is actually a -- dare I say it -- a delightful one…!

That was the first thing. And then the learnings and the insights we gain from that, we’re able to then apply to subsequent framework applications. Digital Outcomes and Specialists (DOS) was the next framework where we really fundamentally redesigned a broken framework. The digital services framework had received quite scathing coverage within the technology press, labelling it not fit for purpose, talking about the relationship between GDS and CCS as being “dysfunctional”, so we had to, and…..Yeah, absolutely. This was escalated to quite a senior level within government, so we had to make a difficult decision which was, we can’t continue with the traditional approach of delivering procurements and contracts in government. 

We needed to stop, we needed to take a different approach. And that was where we agreed with CCS and Government Legal Department that we would form a multidisciplinary team, we would use the GDS design principles as our guidance on how to design and deliver user-centred frameworks and procurements, and using the body of better practice that is the service design manual to basically guide us in all of those things. 

The approach was: working in an agile way; being open; being more engaging with the market to design a completely different framework agreement. And also going live with what we classed as the earliest, what others would say is the minimum viable product, but actually we wanted to call it the earliest usable product. 

This was the buying journey of digital outcome specialist within the Digital Marketplace, so that needed some handholding with CCS to say, “Look, it’s okay to go live with a product or a service that is not complete, because there is no better thing than putting it in front of real users, who are using it for real,” so that we can actually get feedback from those users, by those suppliers, about how is it working. To answer the question: ‘What do we need to change’? 

We could also to use data, as well, analytics from the usage, to inform our iterative service design. That’s never been done before in government, and I’m always proud to be able to say we’re chalking up quite a few government firsts here. 

My next thing, which hopefully we’ll go on to talk about, is just the fact we can iterate and incrementally improve a procurement and contracting experience for buyers and suppliers. This is a massive paradigm shift, I think, in government procurement.

It is, because usually what I’ve seen over the last, let’s say, 10 or 15 years of working in procurement is that once a certain procedure or practice is developed, and usually it’s done organically without any research and looking at what actually works, and without considering the impact it may have on the supplier side, once that is done, it’s done, and it stays there. 

There’s no incentive and there’s no work to revisit what is being done today, to say, “Okay, if you were redesigning it from scratch, how would we do it? And how would you improve it because you now have different technologies, you have different techniques, you can do things differently?” that doesn’t happen in public sector in general. I don’t see this as a criticism more than as a statement of fact. 

So a few years ago, as you know, when I was working with local councils to improve their practice in contracts below the thresholds, that was very obvious, that I looked at procedural, or documentation for the public procurement procedures that you’re following, and you could see it growing up over the years, with new stuff being added without any consideration exactly about what is happening, or what are the reasons for those extra questions or extra elements of fact or whatever to be added into the mix, and

I think it’s very important to get out of that frameset, and instead look at what is being done today, and say, “Okay, do we really need to do this? We really need to improve this.” On that note, bearing in mind that you’ve done a lot of work in simplifying contracts and procedures, have you seen any correlation, or at least relation, between simplifying the contracts and procedures and increasing the participation of SMEs in public procurement?

Absolutely. There is a strong positive correlation. I think, if I state that correctly, that we’ve seen just by simplifying the application process and being more open and describing our intent and what we need, and describing the contractual situation in plain language, I think that we can demonstrate a significant difference between digital services framework, which is the one that was receiving a lot of the criticism, which had something like 150, 160 suppliers on it, across the UK, to Digital Outcomes and Specialists which now has over 1,200 suppliers across the UK, and over 90% of them are SMEs. 

Now, what that is is us creating the opportunity for these suppliers to do business with government, because as we all know, being on a government framework is no guarantee of every actually winning any business, right? But at least we have created the market so that these significant number of suppliers now have that opportunity.

So getting that step one right is really fundamental, because unless you open the door, you’re not going to enable those suppliers to be able to win the business. That’s the first step. 

What we now have to do, and we’re still relatively in the early stages of Digital Outcomes and Specialists, which only went live on the 27th of April, the day before my birthday. We’re approaching three months of usage, but already we’ve seen something like over 160 supplier opportunities published, and we publish those in the public domain, on the Digital Marketplace, because we want suppliers, and not just suppliers, actually, we want suppliers and the public to see, what’s the demand of government? 

What’s the opportunities that, if you’re not a supplier currently on this framework, you’ve actually now got visibility of the demand of government for digital specialists and services associated with the design and delivery of better public services. Having that information in the open is, again back to the open contracting point, is helping to, transform the relationship between the citizen and the state. 

I think we’ll be able to see very soon what’s the actual business that’s being won by suppliers as a result of the changes that we’ve brought about through digital outcomes and specialists. We already have some data about G-Cloud usage. Since its inception, business through G-Cloud has helped the Digital Marketplace go to sales of something like £1.3 billion as it is now. The bulk of that is through G-Cloud, but I think we’ll see it, I’d be very interested, and wait with anticipation, to see what the shift might be from cloud into Digital Outcomes and Specialists, and indeed the growth of cloud as we approach things like G-Cloud 9. 

We’ve just kicked off a discovery to go back and understand how we might need to adapt the design and delivery of that framework based on an understanding of how the changes of the needs within government have happened since 2011/2012, and also how the market has shifted, as well. 

Again, we’ll reuse the approach we took to Digital Outcomes and Specialists to apply a user-centred design approach to deliver the 9th iteration of the G-Cloud framework.

Speaking of what you were saying before, and classified it as a world first, you’re trying to move the government to adopt open data standards, am I correct? So what do you actually want to do in terms of practical changes to the way that the government works, and what do you think is going to be the impact of adopting open data standards in procurement?

What does it mean practically? Well, first it’s great we’ve made this progress. In its open government partnership national action plan for 2016 to 2018, published on the same day as the anti-corruption summit that the UK government hosted, there are very firm and clear commitments around implementing open contracting, as well as a number of other fantastic commitments. 

That gave the backdrop that we can actually then get on and do something. I applaud Crown Commercial Service for making those commitments to implement the OCDS within their operations by October. It’s fantastic, and same with HS2, and beyond that, across governments. 

I saw this as an opportunity to ensure that we’re doing what we can to support those commitments, which is why I’m championing the Open Contracting Data Standard going through the standards hub process right now. Once that’s mandated, that’ll ensure that the OCDS is used across government as an adopted standard across government for open contracting. That’s a practical step to support that. 

Then, as part of that process, we’ll be understanding, “Well, what are the opportunities and the challenges of implementing open contracting, and the OCDS specifically?”. Because we’re effectively designing and delivering an end-to-end buying and procurement and contracting process, in Digital Marketplace we have quite a unique opportunity to embed OCDS and open contracting as a default component of our own platform development.That’s effectively what we’re doing, and we, you know, our ambition is to be the default place for buying digital and technology within government, the Digital Marketplace to be that, and also we can open up our platform for use by others. 

And in terms of another government first, you know we’ve been approached by the Australian Digital Transformation Office to support them with their creation of a Digital Marketplace, so rather than just simply reinventing the wheel, we’ve already shared all of our code, we’ve given them access to our hub repositories, all of our guidance, all of our know-how, and anything that we can share, we have shared. I’m very excited to tell you, we’re sending two of our team over to Australia this weekend, for two weeks, to help the Australian Digital Transformation Office to really accelerate their own development. I’m really excited to see, actually, rather than simply just giving them access to what we’ve done, how might we actually work from a common code base and not just code, but a common asset base for things like the contracts, the design patterns, etc, to really do, government to government collaboration on an area where we have very, very similar challenges and problems around reforming procurement and bringing procurement into the digital age. I think that’s really exciting. We’ve also been talking to the US around similar things, so I see some quite interesting opportunities to collaborate between governments to do great things.

It’s great to see that you’re able to collaborate across borders with other governments, you know, Australia and eventually in the US, as well, but in terms of the rollout in the UK, one of the things I’ve noticed is that there’s a delay or a gap, or even a complete ignorance, to a certain extent, in other levels of government, in comparison with what is being done centrally, in this area, let’s say, of improving procurement practice, in changing procedures, changing contracts, all that and the other. And that, for me, is very, it’s very strange, because coming from a different country where practice is dictated by law, effectively, everyone works in lockstep, so the procedures are very similar in central and local government, or original government, so there’s not too much deviation. So whatever is done centrally, sooner or later rolls out to regional and local governments in Portugal, but to a certain extent that doesn’t seem to be the case here in the UK. So is there anything that can be done to change and improve the rollout of the great stuff that the GDS is doing in other levels of government?

Yes, absolutely, and that’s a really important part of our aim of GDS to transform government together, and work with the wider public sector, not from a position of imposing central mandate or anything like that, but actually being able to reuse what’s considered to be better practices and standards, ways of working, and practices, to help everybody transform themselves into digital organisations, effectively. 

There’s already been some great examples of reuse and collaboration so, for example, the digital service standard, which is, new policy for central government in terms of design and delivery of citizen-facing, and actually internal systems and services, that has been reused and adapted for local government. Now they have their own digital standard for local government, which is, by and large, very similar to the one that was put in place for central government, but, because they recognised that that actually is the way forward, but it wasn’t about us saying, “This is what you have to use,” they recognised that, they took it, they adapted it to reflect the nuances of local government, and they’re now putting that in place.

That’s just one example, and I know that there’s already some interest around looking at things like the technology code of practice, which is again a policy within central government for digital and technology spend controls, which GDS has the delegated authority from Cabinet Office and Treasury to ensure the right behaviours and the right approaches are taken for digital technology spend.

So this technology code of practice is being updated currently, and I think that will be really great to see that reused across the wider public sector, because that’s about helping government to think differently about the way that it approaches its requirements for digital and technology, for service design, for making sure that user needs are at the heart of that, that you’re disaggregating, you’re favouring more competition, from a broader and more diverse range of suppliers, you’re using open standards, you’re making sensible decisions around security, and you’re supporting, effectively, delivery of government as a platform. I think there are some really interesting opportunities. 

There are going to be some interesting conversations along the way. Some of these things are different to some of the entrenched ways of working, and ways of thinking. Helping people to be comfortable with using, for example, public cloud services, you know, more utility-type services, which can be consumed on a pay-as-you-go basis, rather than on-premises infrastructure, takes time. That requires being available to help coach and mentor, and to build the capability and the confidence of people within government and wider public sector who aren’t used to this new way of thinking and working. That’s very important.

One final question. I saw last week a blog post written by you and Jason Waterman from the Crown Commercial Services, about working together to simplify the contract language, and the way that contracts are drafted in public procurement. Can you tell us a little bit more of what you’re trying to achieve, and what are going to be the next steps with that?

Yes, sure. This is, I think, one of the most exciting things that I’m involved with currently, and will be involved with going forward. This is an opportunity to fundamentally transform contracting and procurement within, initially, central government, but I think once we go through the project we’ll see the wider use cases. So Jason Waterman, who’s responsible for policy delivery within Crown Commercial Service, he came to me with a, kind of, a plea for assistance, let’s say. 

Having seen what we’d managed to achieve for digital outcomes and specialists, in terms of a very different way of working, and an output which was a framework agreement that was plain language, it was HTML, I mean, imagine a contract in HTML rather than .docx [laughter] which conforms to the GOV.UK style guidelines, which are very strict quality control to ensure that what gets published on GOV.UK is actually clear, simple, etc.

Very briefly: the process we went about to design and deliver that framework was we had content designers who are actually holding, let’s say, holding the pen, in a digital sense, who are writing the content, but taking input and working alongside procurement and legal subject matter experts to ensure what was created was correct. 

So rather than it being legal or procurement people who were holding the pen, it was people who were trained in designing content. So that gave us a very interesting opportunity to think very differently about the structure, the form of the language, the layout, the usability, the interaction of a government framework agreement. Now, this is not to say that what we’ve delivered for digital outcomes and specialists is the finished product, it’s the nirvana, but what we have done is prove a point that you can apply a very different way of thinking and working, of user-centred design and thinking of the thing as a digital product. 

It’s for the internet, it’s of the internet, it’s applying the digital age, to an area of government that has yet, really, to be transformed and disrupted in this way. To go back to one of your comments earlier, Pedro, is these contracts in government often, accrete over time, based on, case law or whatever. They may have been through a legal challenge and therefore a successful upholding or whatever it meant that we need to do things in a certain way. 

I wanted to introduce the idea that actually we can challenge that in a sense of, “Let’s understand, what are the user needs of a contract? Who are the users of a contract? What are they trying to do?. In terms of the model contract in government, “What currently is working, and what currently isn’t working?” and you’ll see on that blog post, there’s a picture of a contract, or actually it’s a framework agreement, which has been hacked by the user. 

They’ve gone through it and they’ve stuck little sticky notes into the important areas, and they’ve put handwritten labels on that to help them navigate and get quickly to the points within the contract that are important to them. So why don’t we actually design a contract based on an understanding of what the important points are? 

I know we’ve spoken in the past about, one-page summaries, or whatever, but you know, by thinking of this thing as a digital product, we have a unique opportunity to design something that meets the needs of the buyers and the suppliers. Actually thinking of the users being the people who actually need to work together on a day-to-day basis to design and deliver a contract which actually helps buyers and suppliers successfully deliver projects, or whatever it is that they need to do to have sustainable commercial relationships that meet the needs of those two organisations. 

When we start thinking in those terms, there’s a very different set of opportunities which start, coming to the surface. What about if, instead of thinking of the model contract, which is the usual starting point, what about if we have model terms? 

Those model terms themselves will be plain language by default, but also possibly visual, as well as language. They could be visual to help convey what are often complex concepts in, you know, either intellectual property or liability or exit or whatever, that they can actually visualise as well as describe clearly. 

And once we start thinking of model clauses, once we then start trying to understand, what’s the context? What’s the commercial context that this particular contract is there to meet? How would we then assemble those clauses with an understanding of what that context is? So we right-size a contract based on an understanding of that context, rather than starting with all possible clauses with some guidance around, “Well, you don’t, you take this out,” or, “You make sure this is in,” which means you’ve got a really unwieldy starting point. We actually build a contract based on an understanding of what that context is. 

As a digital product, it also means we can put in, and weave into it, the open contracting data standard, so that we simplify the ability to actually disclose consistent, machine-readable data into portals such as Contracts Finder, or the other ones that are available, or even any web service where actually they can just take this information, because it’s publicly available and it’s consistently machine-readable. 

So really, what I wanted to do within that blog post with Jason was just set out what we envisioned is something that’s going to be very, very transformative across all of the common goods and services of government, not just digital and technology. We’re going to be recurrently building the team, and we’ll be kicking off a discovery and then going through the design phases of the service design manual of discovery, alpha, private beta, public beta, then into a live service, but even when it’s live it then won’t be sitting still and stagnating, it will be subject to continuous improvement based on user feedback, user needs and data.

Yes, I think that’s a great project, and something that is going to take some time and energy to get going, and get it to the level it needs to, but I totally agree that is an area that needs to be looked at, and I was listening to you, and effectively immediately came to mind something that has been used for ages in international contracts, or international trade, which is what we call Incoterms, which is effectively a term that contains a clause which is very  clear and very detailed and very stable, so that when you say that whatever you’re buying is subject to Free On Board, which is the name of the Incoterm, you know exactly what it means in any context. So, in effect, to a certain extent, I’ve written about that, on my blog, that one of the things we should be doing in procurement, instead of having only the CPVs for the procurement procedures is having something similar for the contracts and the contract delivery, so some sort of Incoterms for procurement. So I’m really, really excited that you guys are looking into this, and perhaps that is an area that you could explore as a starting point, to see what is out there and what is working already, in a completely different area of work.

Thank you, yes.

We have to finish, it was a great, it’s a shame, I mean, we could have kept on talking for another hour or so, but we need to close it down. So thank you very much for accepting the interview request and giving us almost an hour of your time for a really, really entertaining and engaging talk.



#23 - Mari Ann Simovart (University of Tartu)

Article 72 of Directive 2014/24/EU and limitations to contract modifications


Interview with Mari Ann Simovart from the University of Tartu about Article 72 of Directive 2014/24/EU and limitations to contract modifications. For her PhD Mari Ann looked into the relations between national private and EU public procurement laws with regards to public contract terms, modification and validity. 


In terms of the, the topics that we’re going to be talking today, it’s going to be modifications to public contracts so let’s start by trying to analyse, or describing at least, why do you think that it’s important to limit modifications to public contracts?

Well, to put it very simply, the situation is that if the parties to the public contracts, the contracting parties, had an absolute unlimited freedom to modify the contract as they would have if it would be a simple private contract then it would in fact make all the preceding public procurement procedure a complete joke, yes? Award of public contracts, as long as they are subject to the EU law, we know is always subject to the general principles of transparency, non-discrimination and pursuit of competition as well as elaborate specific rules that govern the process of awarding contracts. So as long as we agree that public contract must follow these principles and these more detailed rules, then they have to have certain restrictions to modification of the public contracts as well, otherwise all the efforts that we have before entering the contract will become futile and as a result there is no transparency, no equal treatment of tenderers, no competition that can be guaranteed in the process. Now, the other question is the level of these restrictions. While we agree that we can’t have public contracts freely modified whenever and however the parties might decide, it is somewhat difficult to draw the line where we have the modification that does not yet harm the transparency or equal treatment and where we have the modifications that can cause that harm. So this is the difficult part but necessary part of regulating contracts modifications. 


If you think about it, it’s almost like saying that we need to limit the modifications to the contracts because otherwise we would be subverting the system or at least allow the system to be subverted during the contract performance which is to actually perform a contract which was not contracted originally.  

Yes, I think you put it very well. That can undermine the system very easily.


But if you look at the level of restrictions in modifications, and I used to be a lawyer in this field and I worked a lot with complex infrastructure projects, it’s very hard to actually get everything right in the first attempt and from the beginning so there’s always the need to modify a contract as you keep performing it. Now, the point you raised here in your first comment is, or in the second comment, that you’ve made is that we need to know exactly where the modification actually harms the legal system and where the modification does not harm the legal system, so where do we find that boundary?

Yes, well, happily for us the Court of Justice of the EU has drawn that line and they have used the criterion of materially different modifications. What was said in the case of Pressetext was that when a contract is considered to be materially different in character from the contract that was originally entered into and when the modification thus shows the intention of the parties to renegotiate the essential terms of that contract, then we have to look at the modification not as just a change to the initial contract but in fact like a new contract. And as always a new public contract requires a new award procedure. So the criterion of material difference or substantial modification is what we have to focus on. 


And in your experience how easy and, or how difficult in the other sense, is it to actually gauge if a contract modification is materially different or not?

In my experience, when I was a practising lawyer, it is very difficult of course. It cannot be easy and that’s part of why I am very happy to have the new EU directives that actually try to elaborate these criteria in more detail.


Yes, because Pressetext is a case from a few years ago and predates the current directive from 2014 so if you look at the directive and if you look at Article 72 of the directive in terms of contract modification, what criteria will we find there that will help contract authorities guide their, the way that they manage the contract performance? 

Yes, well, Article 72 of the new directive, as I understand it, makes three different categories of modifications or three types of cases. Firstly, there are these cases where the directive says: these modifications are acceptable, you can make them without a new award procedure. These are the cases that we consider to be not substantial, or not material. Then if the modification that the parties want to make, or have made in case of a court dispute, if the modification does not correspond to any of these acceptable modifications, we have to look if it is maybe among those that are prohibited in Article 72, Section 4, I believe. And then there is the third type of cases that do not correspond to either of these types precisely and these cases we have to evaluate. So this is the part where the law requires to act as it does now. You have to evaluate the case. Now, the examples of the accepted, or acceptable, modifications one can make. Some of them are easier to establish, for instance under the de minimis rule, we have certain minor changes that are below certain financial thresholds. This is quite easy to establish and in this case you do not need to have a new award procedure.

Then there is Article 72, Section 1 (a) that allows to have review clauses in a public contract if those review clauses are clear and precise so they foresee exactly under what circumstances you can make what kind of change, then you can use these review clauses and make these changes if it is sufficiently clear - you have foreseen already at the beginning that there might be an occasion where you want to change the contract. The simple example of using the review clauses could be maybe an index clause. That’s the simplest example.

Also, the directive allows you to have modification for additional works or supplies under certain conditions and if they remain under certain financial value. Then you can make modifications if the need for that change has been brought about by outer circumstances that a diligent contracting authority could not foresee. Again, that must be under certain financial value and it cannot change the overall nature of the contract. For instance, you cannot say that because of unexpected circumstances you have to build, I don’t know, a school house instead of a hospital. I believe that would be outside the overall nature of the contract. And then you can replace the contractor, again under certain conditions. For instance, if that is necessary due to  corporate restructuring - take-over, merger -, insolvency or other type of succession cases, so these could be the examples where making the modifications has, has been made a bit easier.


A bit easier by highlighting exactly when they can happen. On the financial threshold side, I mean, I have to be careful with what I say here because I’m still bound by my ethics obligations as a lawyer in Portugal, but it’s very interesting because one of the things I saw in practice was that with infrastructure contracts, effectively there was a race to the bottom in terms of the price that contractors or economic operators would offer to do the job, obviously because it’s a winner takes all market and if they’re not competitive on prices it’s very unlikely that you’re going to be able to win the contract. And once they won the contract and once they started performing, then the incentives changed and turned to the other side which is they always tried to get as much money back from the contracting authority as possible so it was always a game of cat and mouse between what they could get extra in comparison with the original price. I remember knowing about a specific case where an economic operator was performing in the contract and the person that was leading that project actually had specific objectives on his contract that if he got more money than it was legally available, he would get effectively a good chunk of cash for himself. This was obviously beyond what was legally admissible but what became very clear over the years is as long as you knew how to do it on a steady way and you used any material changes to your advantage, so for example, the weather, I mean, I remember being on a contracting authority side and for the years that I was in Portugal every single year was the most, or the rainiest year over the last century, which is incredible, at least for the purposes of, the amount of rain that we usually get in Portugal, but one of those reasons that the economic contractors were claiming as justification to get more money out of the contract.

So even though we can actually create these legal rules which are clear and they are an improvement in comparison with the past, I still think that we have an issue with the enforcement and oversight because really difficult once the contract is being performed and if the economic operator knows how to do and how to play the game, it becomes really difficult to actually know what is happening and to detect any infringement to Article 72. But if you think about it in terms of the way that the criteria for modifications are set, it’s one thing what we have in the law and it’s one thing what we have in Article 72, it’s another thing what we have happening in real life in practice. So I remember when I was a lawyer back in the day in Portugal that it was very common for economic operators to submit bids that were unrealistically low at the tendering stage with the objective of actually winning the contract and then doing the performance of the contract to maximise the amount of revenue that they could get from it. Naturally, there were limits as there are now in the directive, there were limits under the Portuguese law of how much money you could try and increase the contract to but I know for a fact that in some circumstances even the project managers in certain projects would receive a nice bonus if they were able to actually bring the money or bring the total money above that legal threshold. You can question, is that legal or not. Probably it wasn’t legal but economic operators became very good at masking these kinds of operations in the sense that actually they do not trip any wires. In consequence, I mean, one of the important issues that we could perhaps look at is the difficulties of actually taking what is in the law nowadays, taking what is on the directive, and actually trying to apply it in practice and ensuring that there is oversight in terms of contract modification and the limitations that there are to contract modifications.

Yes, I know what you mean. I think the example of unrealistically low bids and then ordering additional works or supplies is a very common problem, at least, we have this problem very seriously in Estonia. In this light, well, I was a little bit disappointed in the rules of the directives that said that threshold for the acceptable amount of additional works for instance is so high as 50% of the value of the contract, so yeah, that can be criticised.

But of course generally speaking the oversight or the enforcement part of these rules is very important and it seems to me that right now it is a bit problematic. It has not been addressed by the latest changes to the directives or by the new directives. The oversight or the enforcement part of public procurement rules should be working through two types of means and that would be of course remedies or peer review on one hand, and then of course national public law remedies or the overview by administrating bodies on the other hand. And as I see it, the peer review part or remedies that are used by competitors has been the primary resort in public procurement law until now because most of the remedies have been used for breaches that took place before the award of contract. Now that we have in the directive the rules that concern contract modification, meaning rules that apply after the award of the contract, I don’t think these remedies work so well and that might mean that the public or administrative oversight, administrative review, should have a stronger position.

Or then it might mean that we need to seriously consider if the remedies directives need a review as well because at present all the remedies, all the review system, has been introduced based on the presumption that every breach takes place or at least most of the breaches take place before the contract is awarded. I think the review system works quite well for these breaches but they might not work so well for the breach of contract modification rules. For instance, as I understand it the primary remedy when contract is modified in a way that it shouldn’t be, the primary remedy seems to be contract ineffectiveness and there are lots of questions that come up if someone tries to use that remedy. Firstly, there is the issue of having information about the contract amendment. I don’t think that the competitors generally know what is going on in contracting relations between the competitor and the contracting authority. So there is not so big a chance that they will learn of the contract being modified at all or, even if they learn that, that it has been modified in a way that it shouldn’t be. So one can argue that perhaps the general principle of transparency obliges the contracting authorities to make such modifications or perhaps any decisions made in the performance phase to be public or at least to be publically available if you ask for them. But I don’t think it has been enforced that way, at least it hasn’t been in Estonia. It’s not generally known if somebody modifies a contract and if the modification is made, what are the terms of the modification. So that would be the first obstacle to using any remedy. It’s worth noting that for contract ineffectiveness we have a very strict limitation period - it’s a maximum of six months from when the modification was made. So I don’t think it will be used very often in case of these breaches.


I mean, we can also say that in addition to transparency we’ve got an issue, and I relate it with incentives, which is even if it’s public, who has incentive to actually challenge the contract modification? Because even if competitors get wind that a contract has been modified beyond what it was supposed to be modified, they’re going to be engaging the same practice themselves in the contracts they win so they may not be interested as a matter of fact, in a sense, to challenge any decisions or put the spotlight in those kinds of decisions because they may be on the receiving end in the future. So it may well be that they reserve all their, let’s say all their fighting spirit in a sense for the contract award phase and they will challenge that if they think that something was done wrong but once the contract has been performed, the contract has been performed, they move on to something else. So I think there’s an issue with incentives there. And if you take economic operators or potential competitors out of the equation, then that effectively just leaves us with, you know, the average citizen that maybe having a keen interest in a particular contract, would they go down the route of actually challenging it? I mean, in Portugal we’ve had a few challenges because the Portuguese law allows pretty much anyone to challenge a contract award and also a contract performance, but that’s pretty much it.  

Yes, I quite agree with you. It’s absolutely true that the competitors often don’t have the incentive to challenge contract modification cases. And speaking about average citizens taking control of contract modifications, there seems to be another issue. Even if we find these citizens that would be willing to fight for proper enforcement of EU public procurement law, then in many countries they wouldn’t have a standing to these claims.



So that again leaves only the option to make a claim to the relevant office, the administration, who could maybe, maybe, look at these issues through administrative review.


Or the public prosecution office or something along those lines depending on the, on the country.

Yes. Of course in more serious cases the public prosecutor, yes, that’s true.


On that area as well, I mean, one of the things I think Article 72 brings to the table is actually now there is maybe an incentive for contract authority to declare the changes to the contract and that this was a theory that was presented by a lawyer at a conference that I was recently at and I don’t necessarily agree with it, but there’s now an incentive to declare and make public all the contract modifications that are clearly legal because they will be protected under some type of safe harbour rules. That is true for the ones that are legal but it doesn’t solve the problem with the ones that are illegal. When it comes down to, I think, Article 72 contract modifications if they are declared and they are clearly under the scope of Article 72, that limits the possibility of them being challenged and they are protected by some sort of safe harbour provisions. But I still think that the incentive will be, even if there is an incentive to actually comply with that obligation of publishing contract modifications on an electronic contract notice on the Official Journal of the European Union, any of the ones that are going to be clearly illegal, the contracting authority and the economic operator will still have an incentive of not making them public. 

Yes, of course, of course, naturally.


But that’s my point is I don’t think that you’re going to have a big change in terms of practice because it just protects what was already legal or what is legal under the law. Anything that is illegal will have yet another incentive to be kept under wraps.

Yes, I agree with you in part. Perhaps it will make a change or the new directive will make a change in the meaning that those law-abiding contracting authorities who want to make legal changes only, it is easier for them now. They don’t have to think or evaluate the changes so much by themselves. It’s made easier and it’s a good thing.


Do you think that will actually lead to a change in practice and make, improve at least the transparency of the whole process?

That might depend on the national laws. For instance in the case of Estonia, the new directive will make a huge difference because our regulation earlier was really, really restrictive. We didn’t allow making many changes at all, so now that there is a list of acceptable changes that is quite reasonable in fact, life for law-abiding contracting authorities and suppliers has been made much more efficient and easier. But of course in the countries that already had basically the same rules, there is really no difference.


In a sense, I mean, in terms of your career you’ve been over the last few years focusing on Article 72 and also contract modifications in general in EU public procurement law, at least in contract performance. What are you planning to do in terms of future research in this area?

Yes. In terms of future research I am slowly moving forward from Article 72 to Article 73. So, Article 73 claims that contracting authorities have the right to terminate public contracts in several cases, including the case when an unlawful contract modification has been made. To me, that right contains several fundamental risks, yes?

If you look at the Article, it gives a very broad right to terminate a contract that has been unlawfully modified and doesn’t restrict the option to use this right, doesn’t require any proportionality, doesn’t give a term for using this right, so the risks that we have here is firstly I’m afraid that contracting authorities might in a way start to abuse that right. We have the general principle of proportionality that is supposed to restrict using your rights in a manner or, or in a time that is not in balance with the purpose of the right but if a national law just transposes the Article as it is without restricting it further or prescribing any way to balance it in the actual situation, that might cause problems with regard to proportionality.

Then the other issue is that if we look at the traditional private law principles of good faith, legal certainty, then the right to terminate a contract really has a serious issue because under Article 73 the contracting authority is basically entitled to terminate a contract because of unlawful behaviour that it himself did, yes? Because of [its own] breach of law. So that’s quite a conflict with the principle of good faith and as well with legal certainty.

Then further, if the termination is in fact carried out and the contract is ended, what will happen to the contracting partner, the supplier or the contractor? Are they entitled to compensation and if then to what extent? Or should we regard them as being also a guilty party in this situation? This is another question. So I think there are a lot of fundamental issues that relate to enforcing Article 73 and that would be my next research project.


I think that’s a very good idea and research area to focus on in the next few months and years to come. Mari Ann, thank you very much for spending the time with us in the programme.

Thank you for having me.


#22 - Willem Janssen (Utrecht University)

In-house contracts and public-public cooperation issues under Directive 2014/24/EU


Interview with Willem Janssen, Lecturer and PhD Researcher at the Public Procurement Research Centre of Utrecht University and Twente University. His PhD researches the influence of EU public procurement law on the performance of services by cooperating public authorities and the issues of regulating and enforcing the make-or-buy decision of public authorities. 


Thank you very much for accepting to be interviewed for the podcast. You were selected a few months ago to come to the conference but couldn’t join us so it’s great to finally be speaking with you.

Exactly. And, I mean, let me begin by thanking you for the opportunity to discuss public-public cooperation with you today. Before we begin, I must say that your podcast series is truly a great project. I think it really personifies what academia is in today’s society. I mean, writing a paper, presenting it amongst peers is simply not sufficient anymore and I think research must be relevant and accessible to society and Twitter, LinkedIn, blogs, they can play an imperative role in that to get your message across. But, truly, your podcasts they take it to the next level and I hope that will continue for a while.


I certainly hope so. And I think this series is going to be twenty episodes long, this is the last one, but thank you very much for the very kind words as we start the podcast. So let’s talk about in-house contracts. Why do you think discussing in-house contracts is important?

Truly, like what I said just then, it’s I think the perfect question to kick off our talk today. as I said, I believe the ‘why’ question is very important and the debate on the in-house doctrine and more particularly public cooperation is important for a number of reasons. But, maybe before we delve into more detailed conversation about the particularities of the Court’s jurisprudence or their codification in the 2014 directives, perhaps it’s good to first briefly outline the basics of this doctrine, which is quite technical, but if you’ll allow me, I will aim to briefly sketch their contours.


Of course.

I think it’s important to consider that the basic notion of the EU public procurement rules is that they’re only applicable if a contracting authority awards a public contract to a separate entity. Now, this means that a contracting authority itself is always allowed to provide services with its own departments. Now, this type of delivery falls outside the scope of EU public procurement law and outside of the treaties. This would for instance be an IT department of a municipality that creates and maintains its own digital infrastructure, or a waste management department of the same town that picks up your bag of rubbish every Monday morning.

Now, where it becomes difficult though is when things aren’t as straightforward anymore as these situations, so the question really is, and that’s the question that the in-house doctrine raises, how do these rules apply within the public sector when contracting authorities decide to work together for the delivery of their tasks? And this is, I think, where the in-house doctrine starts. It all begins with considering that there’s no express exemption for contracting authorities that want to provide services in cooperation with others. This means that the rules on transparency and equality that we hold dear in the public procurement directives in principle apply to contractual agreements between two contracting authorities or between a contracting authority and a, for instance a privatised authority which was previously part of the contracting authority itself. Now, this also doesn’t mean that all of these contractual agreements are public contracts that must be put up for tender. Even if they qualify as a public contract they can be exempted from a duty to tender. And I think, it’s important to briefly touch upon three of these alternatives that can exempt a public contract from this duty. 

The first one is the oldest possibility to do so is to grant an exclusive right to another contracting authority. This is currently vested in Article 11 of the new directives. Now, I won’t go too much detail on this but I mentioned it because it was the inapplicability of this exemption in Teckal in the year ’99 of the last century that made the Court of Justice decide to lay the foundations for the in-house doctrine. In this case, the applicable directive on supply contracts did not contain an exclusive right exemption at the time, which did exist in the directive on public service contracts, but as a consequence the Court decided to introduce two criteria, which would deem the award of a public contract exempt from the scope of EU public procurement law. Now, on the one hand, the awarding authority has to exercise control over the receiving entity. This control has to be similar to the type of control, which it exercises over its own departments. I think this is where you clearly see the link with the discretionary power of public authority to provide services with their own department and, on top of this control criterion, the controlled entity also has to perform the essential part of its activities. So I suppose briefly said formal independence must be trumped by practical dependence. A person at a conference once said “just like the Spice Girls, When Two Become One”. [Pedro laughs] So since then numerous cases before the Court have further clarified these two criteria and I believe what has caused a lot of confusion is the different names that it has gotten. It’s been called quasi in-house, vertical cooperation, or institutionalised exemption. Me myself I prefer ‘institutionalised’ as it actually shows that the cooperation is institutionalised in a separate entity, but I suppose that’s really your own preference.

Now, I’ll quickly go onto the second, or the third really, exemption, or the second exemption that was created by the Court in 2009, when it was faced again with a scenario where it deemed these Teckal criteria inapplicable, but it decided to introduce another type of exemption, in an infringement procedure of the Commission against Germany. Now, in this case, the Court ruled that a contractual agreement can be exempted if the cooperation between contracting authorities which does not rely on a separate entity is governed by considerations and requirements relating to the pursuit of objectives in the public interest and that the principle of equal treatment is respected so that no private undertaking is placed in an advantageous position when compared to its competitor. Now, this has since been referred to as the non-institutionalised exemption or horizontal cooperation.

So, I think, these are the basics of the in-house doctrine and now to get back to your question after what I realise was a fairly long introduction. So, why is it important to discuss these exemptions and why is it a good development that this line of case law has been codified in the directives? Well as I said, there’s a couple of reasons I think. I believe the societal relevance of efficient and effective service provision by contracting authorities which aims to ensure that best value for money is achieved is probably the most important one. I mean this means that rules should enable cooperation on the one hand and ban it if a public contract should be tendered in light of the internal market.


Value for money is not one of the directives of the directive and as you said public-public cooperation contracts are not covered by public procurement law because technically there’s not a contract being awarded, so not going to the market?

No, exactly. You’re absolutely right. I totally agree that on the European level the directives focus on achieving the internal market, however on the national level we do have to deal with these type of arrangements and that they do limit to some extent what contracting authorities can do for society or how they organise their service provision. So, I suppose yes, on a more dogmatic level, I completely agree with you, however on a national implementation level we’re still faced with national goals which are often to achieve best value for money.


That’s true.

Yeah. So, I think, the second reason is that there’s a strong regulatory dilemma, and I suppose that also comes back in what I just previously said, is that in EU public procurement law and the in-house doctrine we aim to strike a balance between on the one hand allowing service provision by contracting authorities who wish to cooperate for the provision of services, and on the other hand we try to achieve the main objective of this field of law like you just said which is to create an internal market for public contracts.

So clarity is very important for that and I think that has often lacked in practice, which is particularly troublesome if we look at the Court’s move to apply these exemptions to other subject matters such as concessions or perhaps even more in the future to limited authorisation schemes. And this is I think why I also welcome the codification of the in-house doctrine in Article 12 which in itself should be seen as a milestone of EU public procurement law and all of this is even more relevant when we consider that we’ve seen a rise in the popularity in many EU member states of these type of cooperations which are often in the form of shared service centres or other types of cooperations.


Going into the directive 2014/24, why do you think that the codification occur and why do you think that we have so many exemptions on Article 12 nowadays?

Yeah, that’s a great question actually. I love looking at the sometimes ungraspable regulatory part of the EU. And sometimes things happen for no reason I think. Sometimes they do fortunately but what’s interesting here though is that, what is very clear is that from the start of the reforms of the internal market from say 2010 when former EU Commissioner, Monty at the time, placed public procurement at the centre of his vision for the future of the single market. The new framework was aimed to improve the efficiency of public spending by simplifying the existing rules and aimed for further flexibility when applying them and I think in light of this it was the legislature of that deemed it important to clarify that the rules were applicable to in-house procurement but what we see in the expansion of these exemptions is that it’s driven by a strong call from contracting authorities all over the EU, who clearly found their way to the Commission and to the European Parliament who wanted more flexibility and space to provide services together. And I think that’s what mostly caused these exemptions to expand.


So  changed in terms of public-public cooperation between 2004 and 2014? Because if you look at the way that public-public cooperation is included in the new directive is very different from what we had in 2004, so what do you think changed between these two sets of directives?

So, when they were attempting to codify the Teckal ruling in 2004 it failed obviously. We do now have the inclusion of Article 12 as something unique, in the sense that we didn’t have it before, but of course we had the exclusive right exemption. But perhaps this attempt failed at the time, because either the member states were not able to construct a common wording for this exemption as there had not been a lot of clarifications from the Court following Teckal and, I suppose, this can be explained again by the organisational differences between public-public cooperations across the EU. It would be great actually if everyone did things in the same way which means that making legislation would be a lot easier but, unfortunately, and I suppose luckily we’re not faced with that reality. On the other hand, the European legislature may have been hesitant like I said before to codify the jurisprudence at a stage where many questions and issues had not been decided upon. 

So, what changed afterwards? Like I said I think the emphasis of public authorities in their role of contracting authorities wanting to jointly provide services grew substantially and I think because of that the boundaries of EU public procurement law also became more visible. And I suppose it made contracting authorities and competitors on the market who filed proceedings before the Court more aware of those limitations and with that came the call for more flexibility and legal certainty. 

And this is interestingly enough also reflected in the case law of the Court of Justice. The Court was very hesitant to accept any of the cases put before it until 2007 when a clear turning point was visible in Tragsa and Coditel when the Court became more willing to accept and expand these exemptions. Now, in the past, it’s been argued that this was partly due to the recognition of regional and local self-governance in the Lisbon Treaty. Of course, this is one example of how strongly the EU is changing and that there’s not so much the EU level, or the member state level anymore, or the level of public authorities, but it’s recognising that national administrations are very complex and that we do have a local and a regional level that is also very important and has a say when it comes to codifying or making legislation.


Where do you stand in the argument about accepting or not private capital in public-public cooperation contracts under Article 12, Paragraph 3, in other words are you siding with the Court of Justice in Coditel or Stadt Halle?

I think, private capital, out of all the criteria that the Court has introduced, is probably the most interesting. I mean, this is really where the state, the market and their relationship really becomes visible and you can clearly see how the Court is struggling to deal with this.

Now, it’s fair to say that the Court has been very hesitant towards private capital and that is somewhat justified. The absolute ban provides absolute legal certainty which is interesting and has some benefits because yes there are risks involved when including private capital participation as the objectives of such parties can be different which would mean that control cannot be exercised anymore, and on the other hand it might favour these undertakings over their competitors which appears to prevent any type of state aid.

However, I do question whether an absolute ban is justified and completely necessary, particularly if one considers the practical and financial need for private capital investments that are very often necessary to be able to provide services at a certain level. And also state aid law if you compare it with this other field of law is generally not as strict. I mean, it requires an assessment of exemptions, even if state aid is found it can still be justified, so this means that I see possibilities to introduce a different approach which would be a presumption of diminished control. So, the existence of private capital would presume that control is diminished and I suppose the subsequent relevant question would then be if this means that decisive control is lost.

Now, this was something that was also advanced by the Advocate-General in Stadt Halle. And this different approach would not necessarily exclude private majority shareholdings but, in that regard, it is unlikely that the minority private holdings would be able to ensure decisive control over an entity. And,  I mean in this respect, the higher the level of private participation the more unlikely it would be that control could exist. And it’s not just the actual percentage, I think that should be taken into account after having assessed the applicable national legislative framework, which would need to be considered as that can differ amongst member states in relation to the rights that types of shareholdings give shareholders in practice and if these rights are substantial, which can for instance be the case with veto and blocking rights, decisive control would be unlikely to exist. And, in addition to national legislation, the assessment would also need to include some of the practical determinations of rights in the statutes and overarching agreements which distribute these rights of shareholders and extend the limits of their rights.

So then, if we look at the new directive what’s interesting that the Court is, or the Court, I should say the legislature has in fact done all of this but only for a very limited amount of situations. And this is where only direct private capital can be allowed if it concerns the non-controlling, non-blocking forms of participation which are required by national legislative provisions and conform too with the treaty, and remarkably, which do not exert a decisive influence over the controlled legal person. So, really, what the legislature has done is just introduce this idea in a very small, on a very small scale particularly because it requires this participation to be required by national legislative provisions. I mean, I haven’t checked all of the states but so far what I’ve seen is that a very limited amount of countries, I think particularly France, would be able to benefit from this exemption but in the Netherlands we sure won’t be able to use this exemption.

So, what I’m trying to say, I think the approach of the Court is understandable, but there are certain ways that we can improve this test, particularly if you consider that the whole control test of institutionalised exemption is on a case-by-case approach and to reject private capital to that extent is maybe a bit extreme.


I have to say that on this occasion I think I’m on the extreme side in terms of rejecting private capital, at least where I stand now. The reason for that being that if we allow private capital to benefit from a public-public cooperation contract or services that is being performed, effectively we are allowing someone in the market, in the internal market to benefit from public money, not as state aid but as service provider or at least by owning a part of a service provider that is providing services for the public good for example. But in that sense in my view we’re no longer talking about public-public cooperation, we’re talking about something that involved private capital and as such at least that provision of capital should be subject to EU public procurement rules. Not because them joining the cooperation will in itself generate any public procurement but because at the tail end of the services that are going to be provided effectively we’ve got the procurement or service that is being provided by the state in one way or another. 

So if we don’t do that the consequence is, and that’s what I fear about this field, the consequence will be that companies will understand that in certain sectors it may be easier to actually get in into those projects this way instead of actually trying to win them outright as part of public procurement contracts and public procurement competition and that may lead to regulatory capture, that may lead to very close relationships between certain economic operators in the state and I’m not sure in my view and from my perspective that we would be much better off in the end. 

So I’m a little bit cagey about allowing private capital to be allowed into a public-public cooperation so I fully understand why we should have this exemption in terms of allowing contracting authorities to cooperate, I mean that should be obviously the case, but I’m more and more reserved and I’ve got more reservations about allowing private capital in in these projects. Because even if you apply let’s say the control test, even if you say that the economic operator, the private element of the public-public cooperation contract does not have a control, what if if you have an arrangement where let’s say most of the profits are derived from that public-public cooperation for the private contractor, you don’t have any control, just the terms of the agreement?


You raised some very valid points I think. When I started answering your previous question, what makes it so difficult is when you start mixing public money with private interests and I totally agree with all the points you raise. What I think is going to be very difficult for future things to come, let me just say I think what’s happened now in the directive to allow for private capital only in certain situations I think that’s very inconsistent. I mean it shows that there wasn’t really a vision behind implementing such a thing. I think it’s clearly been put in by one, or a couple of member states, that wanted to benefit from this so, but if you then consider more broadly on what the benefits of private capital could be for the performance of or the realisation of certain projects I think still that the Court’s approach is too strict. 

And I think some of the points you raised, I mean, if the benefits or if the disadvantages of including private capital are really that strong there’s always the option to tender that type of capital participation to still create some type of competition to open it up to the market, so actually the participation would still be in need of obliging with the public procurement rules which I think would solve a lot of the issues that you raise.


But there’s no obligation in doing that. So there’s no obligation in privatising, because effectively you’re talking about a partial privatisation of a service, there’s no obligation in tendering. And I agree with you in that I think that’s one of the limitations we have in terms of public procurement rules is yes public procurement only deals with actually expenditures or expenses made by the state, so buying something, it does not deal with public-private relationships that may influence internal market if what is happening is money coming in the other direction, so money coming into the state.

No, it just makes me realise how interesting public procurement really is, I suppose. Because we are fortunate enough to be practicing in or studying a certain field of law that touches upon so many interesting subjects in society even though the actual content of public procurement procedures as such and their focus is sometimes limited and, but it does touch upon a lot of other fields of law, a lot of societal relevant issues which makes it very exciting to go to work every day I find.


One final question. How can we solve the legal uncertainties arising from Article 12, Paragraph 4? So what would you do to improve Article 12, Paragraph 4 of the directive?

Ooh, you’ve left the biggest and toughest question for last! [Laughter] I think there’s a lot of people that have looked at this exemption, a lot of people have discussed it and it’s still a very difficult exemption to grasp. And, I suppose, the newly introduced activities criterion appears to be the least troublesome, but it’s mostly because of the use of terms such as ‘public services governed by considerations relating to the public interest’ that cause difficulties and leaves open much room for interpretation. And because these terms differ with regard to national and cultural traditions or at least the way it’s been defined over the years, it’s nearly impossible for the Court to give a clear-cut answer.

Now, I think that practical cases before the Court will undoubtedly still clarify these criteria but only on a case-by-case basis, such as what happened when the Court clarified that cleaning services were not in the public interest in the case of Piepenbrock. But still, there’s a lot of open questions in this regard which I hope that the Court will address in the future. So say for instance the Court used to emphasise that no private provider can be placed in a position of competitive advantage and we cannot find this anymore in the codified criteria so it can be questioned what the actual relevance of this still is. And, more specifically, I think, and that’s where a bit of fundamental reasoning lacks, is how a cooperation, or at least I wonder, which can have private capital participation in the participating bodies and which is active on the market for 20% of its activities can still be governed by considerations relating to the public interest?

It’s a very difficult question and let’s hope that the cases before the Court will come, actually address some, and there’s more of these, uncertainties. And generally speaking, more attention should be paid in Europe on the European level and on the national level to identify the type of services that can rely on these exemptions, or at least how we come to a conclusion that something is in the general interest. This is a very old debate but it’s still so relevant as we still find it in much legislation.

I think, on a more broader level, the potential extensive application of these exemptions in practice raises more fundamental questions of how we approach the in-house doctrine. Mostly because the focus of these exemptions has always been of an organisational nature, how a service is organised depicts the influence of EU public procurement law. This would bring me to my push to have more attention to the phase prior to procurement in which a public authority decides which type of services it deems best for society as that is where the economic and societal relevance lies. And don’t get me wrong, this is a very difficult area to think about regulation, to think about enforcement, to think about how do we improve this, and I think this is also partially why it’s currently a national matter. It’s often left still unregulated also on the national level but definitely on the EU level because it delves so deep into the roots of the organisation of many member states. 

Article 345 of the Treaty, the Protocol (nr. 26) on services of general interest, recognise firmly that national member states have held the discretion to define these services. But, nonetheless, the effects of in-house performance on the market are not necessarily taken into account very often. Innovation, sustainability can be hampered by in-house performance, but it can also be advanced by in-house performance. And what’s probably even more predominant and important from a legal perspective is that third parties have no say in this decision making process before procurement or in-house performance occurs, whereas their interests are often affected by the internalisation of a public contract which means there’s a lack of foreseeability as well.

Okay. I think that’s a great way to end the programme. Willem, thank you very much for accepting the invitation to come to the show and for the last half an hour.

Perfect. It’s been an absolute pleasure. Good luck with the rest of the series.


Thank you. You can find me at my blog or on Twitter where I use two handles, @Detig for general discussion and @publicprocure for public procurement related topics. As ever I’m very grateful for the support of the British Academy Rising Star Engagement Awards which made this series possible.

#19 - Stephan Litschig (Graduate Institute for Policy Studies)

Does an increased audit risk affect corruption practices?


Interview with Stephan Litschig, Associate Professor at the Graduate Institute for Policy Studies, GRIPS, in Tokyo. His research interests are in the area of development, public and political economics. His work has been published in the American Economic Journal: Applied Economics, the Journal of Public Economics and the Journal of Human Resources. He also serves as an Associate Editor of the Journal of Human Resources. Today’s talk is focused on his joint paper with Yves Zamboni called ‘Audit risk and rent extraction, evidence from a randomized evaluation in Brazil’.




I find your paper with Yves Zamboni very interesting and what I would like to know for us to start the conversation with was what led you to carry out this research.

The motivation was that we know that let’s say anecdotally, corruption is very prevalent in many countries but it is much less known what the people could do to reduce corruption. And by corruption what I have in mind in particular is in public procurement for example that public officials will take kickbacks in return for steering the contract to certain suppliers but I also have in mind a broader concept of corruption which is rents in the delivery of public services where the fact is that in many countries public servants don’t show up for work, teachers for example or health service providers. So the basic motivation is just that we know that procurement and service delivery in general could be better and then the question was can we change this by increasing the scrutiny that comes from the top basically, increased audit risk.


And why doing this in Brazil specifically?

So this was an opportunity I had come across the fact that in Brazil they have this peculiar way of auditing local governments which is done at random so there’s literally a lottery that is done that’s actually on television and so that, what that gives you is that it gives you comparable data on audit findings across Brazil, so for a large number of municipalities, and then I basically got in touch with this agency and wrote to them and proposed this and my current co-author was the one who picked it up and said “look, this is interesting, let’s do this together”. So it’s basically the fact that in Brazil they were doing a particular interesting way to conduct these audits and what we did with Yves we designed this study on top of this existing lottery where we added another layer that basically put aside a hundred and twenty municipalities during one year into a high audit risk group so these guys knew that, basically in the control group audit risk per year was something like 5% but if you were in this high audit risk group you had like a 25% chance to be audited over the subsequent year.


How was that selection done? Was that selection done by you or specifically was that a purposeful sampling or was it done at random saying “you guys in this year actually have a higher audit risk”?

Yeah, exactly. So it was purposefully at random if you want so, you know, they were already doing this random sampling using a literal lottery, like it’s the same machinery that was used for a money lottery in Brazil, that was very popular so they used the same machines, then we also used those machines to set aside a group of municipalities that, because it’s a random sample should have all observable and unobserved characteristics the same as those that were not selected. So then the idea is that if later on once the auditors go in and they find differences in the incidence of irregularities of matters of corruption this has to be because of the only difference between the two groups which is that they were exposed to ex ante higher audit risk.


And did they know that in advance, so were they informed?

Yes. So that’s an excellent point. So the overall lottery that was regularly done by this audit agency CGU was very well publicized so in that sense we could assume that they knew about it but to make sure that those that were put into this extra scrutiny high audit risk group, that they were aware of that, in addition CGU also sent them a letter and said “by the way, for one year you’ve been selected, you are one out of a hundred and twenty municipalities and at the end of the year we’ll take thirty of you to be audited according to our usual audit procedures”. So they did also receive a letter.


In terms of communication of information to those municipalities, so you guys sent a letter but if you’re looking into corruption to the municipality you’re looking into probably I would suspect multiple levels within the organisation, so did you expect the information about the higher audit risk to filter from the top down or just for that information to remain with the person or the persons that received it?

So we were agnostic about that, so the idea was that perhaps, so the mayor for sure would receive the letter…


Sure, yeah.

…but then some of the people in his cabinet would receive it, then, whether he would communicate this to, down to the health offices and all the teachers we were sort of, we didn’t want to micromanage that so we just said “okay, let’s just send this letter”. The key thing that changed was that objectively they were in a high audit risk group so we thought about this but we thought we don’t want to make this too artificial in a sense. Suppose the government would increase the audit intensity, right, they’re not going to send around lots of letters, they’re just going to do it, they might inform some people but we wanted to keep this as real as possible. But it’s true so what that means is that we don’t know how far the information travelled, that is a limitation.


In terms of findings, what were your main findings?

So we find that when we look at procurement outcomes we find that there is a reduction in both the share of the amount of audited resources that has evidence of corruption, so both in money terms but also when we look at it at the procurement process level and there’s something like 1,600 procurement processes that we have the data on and we find that at that level as well there’s a reduction in the probability that auditors find evidence of corruption. Now what does corruption mean? We basically use, so these auditor reports have been used before in the literature so we use existing corruption codings and one unambiguous example would be the auditors go in and they find that the firm that, you know, received the contract in fact doesn’t exist. So that [Pedro laughs], it’s scary but these things happen, you know. Other…


I know.

…other things that, right. Yeah, yeah, well, you know, some people are very surprised but I guess you’re more of an expert in this, in this area. But yeah, so other things are when they go in, the auditors go in and they find that the government paid more than what they had agreed to in the auction basically, without gaining any other compensation.


I was laughing because a few years ago in Portugal when we started collating the contract data, some people that I know in Portugal they cross-referenced data from that particular database, the contract database with information from the companies registry and they found that certain companies were being awarded contracts even before they existed. That’s why I laughed because I’ve seen that kind of stuff happening before!



So it’s not only Brazil, yeah.

No, I’m pretty sure. You know, so this, you know, I’m getting ahead of myself here but I think in terms of future research there is a lot to be done here, just sort of, just in terms of measuring how frequent does that happen in different countries in different contexts, I think we just know very little about that. And, you know, one reason for that that we know little about it is obviously data, data problems of merging different datasets but the other one is that in most countries the audit agencies would target sectors or firms where they think there’s a high likelihood of corruption and you end up with a very unrepresentative sample of things of what’s actually, what the state is in the overall country. The nice thing about Brazil is that because of this random sampling what we have here is basically a, you can think of it as a random sample so if we find for example that the incidence of corruption is something like let’s say 40% amongst restricted modalities, that’s one of our main findings, we can sort of confidently say that’s more or less what’s going on in the public sector at a local level in Brazil, as opposed to, you know, this reflects the tip of the iceberg or something like that. But, you know, to do that you need the random sampling and that’s, no other country in the world does that so if you want to do comparable studies across, just, you know, compare the incidence of corruption across countries you have to convince public officials to do things in a random way and that’s often very tough. But that’s, in terms of future research that’s sort of where I think this could go.


That’s one of the point that I would like to drill down a little bit further which is in terms of processes, which processes have you found out that were more likely to generate corruption or have corruption arise from them?

Yeah, so that was an interesting find is that for the first time in our study we have the detailed process level data which means that basically that process is it’s a given purchase, they vary a lot in terms of magnitude but what we know for each purchase is was it done through a modality that restricts competition in some way or not, which is an open auction and you specify in the call what you want and then anybody can pretty much apply. So we group modalities, there’s more than those two but we group them into two and the finding is pretty stark, is that, amongst let’s say public procurement auctions the incidence overall of corruption is much lower, something like 20% and it doesn’t change at all, whereas amongst modalities that restrict competition and give more discretion to the public official, the overall incidence in the control will be something like 40% and there we find the reduction, it only occurs there, something like 15 percentage points. So it’s only the modalities that restrict competition, that afford more discretion that, where this corruption seems to be going on and where it is affected by higher audit risk.


Does Brazil have financial thresholds in terms of determining which kind of modalities you’re going to use, if you’re going to use the auctions or you’re going to use the restrictive competition ones?



Or is it just done by case-by-case scenario?

No. There are thresholds and one of the things that the auditors look at and that we also, that is part of some of our corruption measures is in fact whether they basically fraction the purchase to avoid those thresholds. So there are thresholds.


Because that is fascinating, especially, or even in Europe because the financial thresholds in Europe are quite high so you only apply rules for contracts above 100 and something, for goods and services 100 and something thousand Euros if I’m not wrong or more, or even more if you’re a local authority. And I’ve always thought that it was not a very good idea, especially on corruption grounds but I’ve been writing about that area over the last few years and I’m going to revisit it again and you’ve just given me a new piece of the puzzle for me to work with which is actually where there is more discretion for public procurers we are creating the conditions to generate more corruption or at least make corruption more likely?

So that’s what our findings suggest for the first time that at least in Brazil in this particular context this discretion tends to be abused, okay. Now how externally valid is that? As you say I think that’s…


Yeah, we don’t know?

That’s research, we don’t know, that’s right.


Okay. But in that sense I’m writing about these ideas and obviously this area of public procurement from a legal perspective so I can afford the limited availability of data to try and shape my thoughts, although obviously I want it to be sustained by evidence and…?

Mm, and of course, of course.


Right, okay. So I’ve got a few extra comments for you.

Sorry Pedro, can I, because you asked and I’ve been going off topic I think so the paper is about procurement but if you don’t mind?


Yes, sure.

One of the things we also look at is this idea that when you increase the audit risk service providers are more likely to actually show up for work, okay. So there’s, it’s not corruption in terms of stealing but corruption in the sense of you’re paid to do a job and you don’t do it so, and the reason we can do that is because the CGU audits they include that as a standard procedure for them to basically also try to assess whether for example in health service delivery the officials, and these are going to be doctors and nurses but also going to be health workers that are supposed to visit the health facility users preventively, so the idea is that it’s cheaper to prevent than to cure but to, the prevention requires that these guys go out there and visit the families to find out whether someone is ill so that the prevention can actually take place and that’s costly of course for these agents. So what we looked at there or what the auditors looked at is they ask a random sample of potential service users in a given area they make household visits and they ask “are you receiving these visits from these preventative healthcare workers?” They also ask “when you go to the health post is it open when it should be open and when you go there is the doctor there?” These kind of questions so, and they ask some questions in control and treatment municipalities, okay. Now what we find on that front is that although in the control group there is not perfect performance in the sense that people do report that the health post is not always open as it should, the doctor is not always there, the increase in audit risk has no effect on that performance, on that compliance with your stipulated job requirements. In stark contrast to what we find in procurement, we find this reduction.

So what we, we were puzzled by this, it was we didn’t expect that but then what we do in the paper is we obtained a standard economic model and we see whether we can rationalise these findings. And one explanation for these results could be that when you get audited in procurement basically you can’t argue with those findings, you know. If the auditor finds that the firm that you awarded the contract to doesn’t exist there’s not much more you can say, whereas if the auditor goes in and he finds that people report that the doctors were not in the health centre when they should have been, the doctors are going to say “no, I was there” and the audit finding doesn’t generate that kind of hard evidence that would make people work. So we think that ex post we can rationalise our findings so the fact that health service providers might just not be worried by the audit because they can always say “well no, no, I was there” or “I was in another room” or “I was, I don’t know, I was visiting a family out in the field” so it just doesn’t generate the same kind of evidence. And so that, you know, what that, it just, I guess it also, it’s not, obviously it’s not clear how general the finding is but it raises this issue that the quality of audits can be useful to monitor certain things but not others. And then that has implications for audit design so perhaps the auditors can save these visits because at the end what’s being generated is not really useful that much.

Anyway, so I just wanted to mention that. I know this is a procurement audience but because the increased audit risk happened for everybody in the entire municipality and because the auditors look at these things as a matter of routine we thought that it’s useful to look at that as well, even though it’s a null result we think it’s interesting.


That is interesting and I was wondering as I was listening to you if it may be that the consequences are actually as far as I know very different in case you are found involved in corruption, actual corruption in a public procurement process or just not doing your work properly?

You’re exactly right. So there’s an additional reason which is that if you’re found stealing from any public procurement you can basically go to jail, whereas even if you’re convicted that you were shirking on the job you might lose your job at most.


At most, yeah.

You know, the sanctions are just less as well. So you’re exactly right, that will be another explanation. To be fair there’s also the explanation that as you said earlier, right, information might not have travelled to those guys, could be that the…


They did not know?

They didn’t know they were not aware of this audit thing and for them they were not treated in some sense. And, you know, we won’t know [laughter] so…


I’ve got another question here, a little bit more specific is that I think it’s on page 11 or 12 that you say that corruption or at least stealing less today does not imply that more can be stolen tomorrow, so you argue that because the federal funds cannot be saved for periods longer than the budget year that there’s no possibility that someone will not steal today so that they can steal more the next year because they’re just going to lose the budget?

Uh huh.


I agree with you, I think that is true but I was wondering if you found something within, any difference within the same budget year? Let me explain, what I’ve seen working with public procurement officials in different countries is that sometimes when we get close to the end of the budget year you still have some of your budget left and you need to spend it, you need to spend it and you don’t usually spend it in the best way possible, so that would increase the likelihood that someone would be tempted to take some money or be corrupted in one way or another. So what I’m wondering is if within the same budget year or fiscal year have you seen any differences or potential differences in terms of one is more, when is corruption more likely to occur?

Mm. It’s an excellent question but without data…


Without data, yes.

We are bit limited on that because we don’t know basically when the purchase was decided, you would need monthly data in some sense, right? You need to know when was, and that we just don’t have. I think for other countries, I think for Chile there’s actually a study that basically shows what you’re saying. Maybe you had that study in mind? But so …


No I don’t, I don’t. I would like to know more about it so if you know it just give me the information afterwards and I’ll look it up?

Yes. I don’t have the authors off the top of my head right now but this has been empirically documented. Now I think what they look at if I remember correctly is just, do they, so I’m not sure they actually have audit findings on these transactions, I’m not sure if they can say that it’s more corrupt if it’s spent towards the end of the year but things do happen that are special at the end of the year, that much I remember.


Okay, very well. One final question, next steps and future research plans in this area, do you have any? Do you think that you’re going to still revise the paper one more time or…?

Well the truth is that it’s not published yet so I’m sure I’ll revise it, that’s just, but that’s nature of the game and, you know, in fairness to the referees it usually gets better. But the paper is what it is. What I do find though is that academically people tend to, at least the referees I’ve got, they often say “well it’s not really surprising, we already know that an increase in audit risk will reduce corruption, it’s sort of obvious” and I think that’s just wrong and I think it’s actually, it hurts public policy because you just assume these things and you don’t study it. The fact is that we don’t, we know very little about the incidence of corruption in procurement and service delivery more generally across the world and then what works to reduce it, if it has worked in one context is it going to work in another one? We just have no idea. So I, there’s the, the short answer to this is that I think I’ll continue working on this topic trying to see whether more monitoring or other kind of monitoring or even also incentive schemes can improve service delivery and reduce corruption but I’ll have to find willing partners and that’s, you know, that is the, you have to work, to do this kind of stuff you need connections with governments and that’s, that is sort of the major constraint, okay. So I can say I want to do that, the question is going to be am I going to find governments that are willing to do that kind of stuff and I can just, all I can do is try. So that’s one.

Another area where this can go where it’s a little bit, a different question is the fact that public procurement has in recent years moved very much online so e-procurement where from a monitoring perspective what’s interesting about that is that there is much more information now about let’s say the awarding stage, right, so you can basically go online and see what kind of contracts are available for which you can bid, you can see afterwards, you know, who won, so there’s just much more transparency in the entire awarding stage and I have a project with two other co-authors where we try to see when, under this increased transparency in the awarding stage corruption or irregularities get shifted more towards the execution stage because in order to know whether the price agreed was actually the price paid or whether the quality was according to what was agreed or not, whether it was substandard quality in the execution, you actually, you still need an audit. This whole movement of e-procurement hasn’t changed the fact that we know very little about what’s going on in the execution stage and there’s our worry that just the focus of corruption might shift towards execution. So that’s just, you know, putting sort of a question out there that we’re working on but we haven’t, I don’t have any results but I think that’s, in the future that is interesting, So less or increased transparency at the awarding stage might actually shift, just shift things around and overall towards the execution and that, and overall you might not get much of a benefit. So that’s a worry that we have and we want to investigate that.


That’s a very good couple of points that you’ve made there for us to conclude the show on. I would just like to say that I agree with you that the more transparency that we have during the awarding stage, the more likely it is that whatever, let’s call it irregular behaviour or illegal behaviour taken by agents is going to move elsewhere. You are right in looking at the execution stage because execution stage unfortunately so far has not been subject to as much scrutiny as the award stage and that for me is a mistake. It’s a big, big mistake in terms of policy and I’ve been against that view of procurement, that procurement is only the award stage and not execution for many years. On the other hand something that you may not be aware of is that at least in Europe under the new EU Directives it’s now possible in variable terms unfortunately in my view for contracting authorities to have conversation with suppliers even before they open the award stage, so that means that there’s another, there’s another moment where corruption can occur which is actually before the award procedure. So even if you have let’s say an open procedure, so what you guys would call an open auction for example, if you have an open procedure, very transparent, very clear, you look at it, just at the procedure and you will not find any corruption but you don’t know that for example one of the bidders was told in advance what was going to be included there and for example shaped the actual award criteria.

Right, so the buyer can tailor the call to whatever they, and you’re saying this is legal in Europe?


What I’m saying is that it’s legal nowadays for the contracting authority to have conversations with suppliers before it launches a tender and the way that it is drafted in the directive it’s very, very liberal and way too much liberal for my taste.

Interesting. But so is this a change that’s happening?



Or are you, ah.


Yes, it is a change. I mean economic operators know but contracting authorities have been asking for this for many years, they say that if they are forced to use an open procedure, say an open auction, that is very difficult for them to know in advance what they really need and how to spec it out correctly so they need to talk with the market in advance, so if they talk with the market in advance the procurement is going to be better and I just say bollocks, you’re just creating the conditions for undue influence to occur. But that’s another area that is coming up at least as a potential area of research in corruption in public procurement.

Well very interesting. Thanks a lot for letting me know. Well, you know, let’s keep in touch. I mean clearly I have a lot to learn about what’s going on in Europe.


Okay. Thank you very much Stephan, was a pleasure talking with you.

Thanks Pedro. See you.

#18 - Kirsi-Maria Halonen (University of Lapland)

Contractual ineffectiveness in public procurement


Interview with Kirsi-Maria Halonen, Senior Lecturer in Law at the University of Lapland. Kirsi worked previously for Hansel Ltd, the central purchasing body of the Finnish Government. She concluded in August 2015 her Ph.D at the University of Turku on the consequences of contractual ineffectiveness in public procurement.



Let’s dive in, straight into your research, you finished your PhD last year on contractual ineffectiveness in public procurement, so could you tell us a little bit more about that research? 

Yeah, sure. So I think I wasn’t concentrating that much on the remedy of contractual ineffectiveness.  Of course I looked at the rules concerning contractual ineffectiveness as well but my focus was merely on the consequences of the remedy itself between the contracting parties, so I was interested on whether the contracting authority can be held liable towards the contractor, for example, in cases where they have infringed severely public procurement rules which have led to contractual ineffectiveness. Then I also looked at what kind of liability regimes could be applied on this kind of liability, if there is any, and what are the possible ways to manage risks of contracting authorities, is there a possibility to mitigate the damages somehow and are the rules of contributory negligence applicable to this kind of liability.


So what did you find?

What did I find? I think you can divide those conclusions in three parts. So firstly, I think the most important conclusion was that at least according to Finnish law, Finnish private law, the trust towards the actions, the rightfulness or lawfulness of the actions of public authorities is very strong. As the contracting authorities are the ones actually responsible to comply with the public procurement rules, and if they infringe those rules they can be held liable if the other party is not aware or should not have been aware of this infringement. So there is a certain risk for liability in these cases. And the second conclusion that I found was concerning the liability regimes. If the contract is declared ineffective there is no contractual liability available for the parties so merely the liability regimes that could be applied here are based on extra contractual liability such as principles of culpa in contrahendo or the general law on damages. And then the third part of my conclusions relate to the ways of risk management or liability mitigation so I looked at the limitation of liability clauses and whether these are considered binding and effective. Then I looked also the rules concerning contractor’s bona fide so whether or not and on what grounds we can decide on if the contractor was actually aware or should have been aware of this infringement and what effects it has on the contracting authority’s liability. Also the general rule of duty to limit your own damages is applied here and I concluded that actually EU law doesn’t prevent the application of such rules and therefore it’s based on national law whether or not these kind of rules are applied here.


And did you look only at Finland or did you cast a gaze as well to other jurisdictions?

Yes, my main focus was in Finland and I also conducted my research and published my dissertation in Finnish but as in Nordic legal studies often it’s done, at least in Finland, you also look at the neighbouring countries, the other Scandinavian countries, especially Sweden and Norway where the legal systems are quite similar to ours. Usually in Sweden you can find some quite interesting thoughts in the legislative preparatory works. Then I looked at France because they have a long tradition with contractual ineffectiveness. Also the other southern European countries do have similar traditions, like Spain and I believe also Portugal and Italy have a tradition of nullity of public contracts that may be ordered by a Court, which we didn’t have up in north before the rules of contractual ineffectiveness were transposed. It was interesting to find out that there was actually a previous case law also relating to procurement contracts on these kind of remedies, so that’s why France was interesting. And then I also looked at the UK a bit because there the attitude is very business oriented in a sense that there is lots of freedom for the parties to agree on things, for example, to agree on risk sharing for the event of contractual ineffectiveness which is explicitly stated in the UK’s procurement rules. This has created a lot of discussion on whether or not that kind of legislation could be possible also in other countries. So there were interesting details that I found in looking at other countries.


Why is the interplay between national contract law and public procurement rules important?

 Well I think it is important for the reason that without looking at things on several perspectives you are not finding true answers. So if you are looking at the consequences of contractual ineffectiveness solely from public procurement rules perspective, you are not finding a complete answer. In Finland traditionally public procurement has been viewed as a part of public law, administrative law and as an administrative procedure to choose a winner in a certain contract award. But if we are looking at the rules as an administrative procedure to choose a winner, then we don’t get the whole picture of what happens afterwards because we’re actually buying something, we’re making a commercial transaction and we’re not applying public procurement rules for the reason that we want to just comply with these rules as such but as a mean to purchase something. That’s why I think it is important to also take private law into account, meaning the contract law commercial law in general, as public procurement contract in Finland is considered as a private contract. It is utmost important from a buyer’s perspective to realise how these private law rules should be interpreted in the context of public procurement where EU law, and especially its principles, are actually limiting the interpretation which usually in private law is considered to be a freedom of the parties to agree on anything and here EU law has, and the public procurement rules have a lot to say on how to look at things.


I think that’s a very important point and it’s intimately connected with one of the issues that I’ve found over the years in public procurement which is that there’s a huge emphasis and focus from EU rules on the procedural part of things but when it comes down to the actual contracts, the rules that are applicable to the contract are left for the national member states.

I have not done any research on this, but this is just my gut feeling, my take on this is that in consequence economic operators find it more difficult to actually apply for contracts in other member states because they don’t know what will be the contract law that is going to be applicable to that contract. They know that in terms of residual rules they’re going to be entitled to certain rights and certain protections but even if they’re awarded a contract then they’re subject to national law.

Yes, you are absolutely right and I know that you have a background as a procurement lawyer in practice as well, as I do, so yeah, often also research in public procurement is done via the principles of EU law and the public procurement rules as such. But somehow I think it is very important to always keep in mind that this is the reality, this is actually business and we need to look at public procurement rules as a toolkit to get the best possible result. And here, as you pointed out, if we are wanting to create a single market then it should be clear for all the players at the market what are the common rules and where comes the line after which the national rules are applied. And this is not clear for me, not even in Finland, nor I think that it is necessarily clear in UK or Portugal for example, If you think about private companies, I believe that this might create some obstacles to participate for sure.


Yes, indeed and just by coincidence yesterday I was watching a programme on Euronews about the strategy by the commission for the digital single market and one of the actual legs of that particular stool that the commission wants to set up was to uniform or have some sort of harmonisation at least for contract law that is applicable to cross border online contracts. And if you think about it in general, online contracts or online contracts for consumers, involve very low amounts of money so if the commission understands that for that specific sector it makes sense to uniform or at least harmonise contract law. Why aren’t we thinking about the same for public procurement?

Oh that’s a very good question and I, unfortunately I don’t have the answer to that question but I think that that could really speed up the progress of opening the borders in public procurement. One other thing that comes to my mind relating the differences of contract laws in different countries is the binding effect of an offer. So basically there are different rules in different countries on the binding effect: a company submitting an offer to a contract award in Spain might be able to cancel it if it wants to but in Finland I am not able to cancel a bid after it has been opened. In Finland actually according to the basic contract law rules, not though in public procurement anymore, a contract is considered concluded when the offer is accepted but already before this moment the offer itself is binding and it can lead to similar consequences, similar liability issues as if you would cancel a contract.


That's a common discussion in contract law in various jurisdictions which is each jurisdiction has defined its own way of interpreting the moment when the contract is formalised, so is this acceptance or the sending of the offer, if the offer contains all the necessary elements, so on and so forth. But again in the field of public procurement that is one of the areas that’s perhaps at a level we have not spent enough energy and attention in trying in sort and solve. On that note what do you think should be done at a level in the future that we could consider to be a potential solution for the differences that we are finding in the treatment of contractual ineffectiveness in public procurement in the various member states?

 Well I think it is important that EU public procurements rules don’t go too far in limiting the applicability of national public or private contract law or national public procurement rules, but at the same time we should have some clarity on which rules or on what aspects EU law are applicable. So that could help the national systems to develop their own legislation and argumentation relating thereto and to provide similar results even though the legal systems behind these rules are quite different. So I would want to know what are the actual conditions of liability for example in certain issues. I would also like to know whether EU law is actually giving any protection to ex-contractors when the contract is considered ineffective. I think now the European Commission’s point of view is that there is no protection provided towards ex-contractors because they have participated in the infringement even though they wouldn’t be aware of the infringement but it is assumed that they must have been aware of it. I think there are questions concerning remedies and then there is also a question of clarity so that the companies involved could really know what are the conditions of liability and what is the proof they need to provide in this context? So it wouldn’t always be a matter of national law to interpret, it’s according to their own principle. But as an answer to your question, I think there are several small things that could be developed but I’m not saying that something should be done or something shouldn’t be done but at least we should know what are the exact rules that are applied.  It is a bit ambiguous at the moment where goes the line between EU law and national rules? But, at the same time, this is very interesting from a researcher’s point of view, so if we bring here too much clarity then probably I need to get another hobby.


Do you reckon that in a future review of the remedies directive perhaps we’ll see something along those lines or do you reckon it’s not going to happen?

I think maybe, though I think now it’s all about principle of effectiveness, especially the length of judicial procedure that is on the table. I still think that EU legislators are not that worried on the parties of the contract, the ex-contractors and their rights, as the interest is focused on removing unlawful or infringing decisions and bringing back the contract to the markets which is what the remedies are mainly designed for. The main focus lies on the remedies for those who have suffered from contracting authority’s infringement and not the rights of those who have been part of it. I think those companies are not going to get a lot of support in the remedies directive in future either..


I think it’s a shame because as we were saying before, if you don’t tackle some of the outstanding differences in terms of contract law, in terms of consequences, I feel that uncertainty, the uncertainty that is generated by these different systems, actually probably deters economic operators from participating in the first place and obviously if they don’t participate they cannot win the contracts in other jurisdictions, in other member states. So I think it’s a shame that the discussion is still very much focused on what I would call formalistic approaches to remedies, saying “Oh do you have access to the Courts and only the Courts and nothing else, do the Courts decide quickly enough?” That is important but at the end of the day that is only one part of the discussion.

Yes, yeah, yeah, for sure, I agree with that.


But we are where we are.  



Speaking about research and speaking about interest, you spent a lot of time over the last few years looking at this interplay between public procurement and contract law, looking at the contractual ineffectiveness in public procurement, where do you want to focus your research going forward?  

Well as you said, I have been quite involved in public procurement and private law matters, I really like it, so I don’t plan to entirely abandon it but lately I have been more and more interested on the requirement of transparency, quite classical subject I would say. But as a broader phenomenon combining lots of different goals and objectives, maybe controversial towards each other, either fighting corruption, securing the access to remedies in a public procurement process and at the same time its potential adverse effects on creation of innovations as a commercial interest protection issue. And perhaps we will see an empirical research relating to these aspects during the few years. And I’m also currently discussing with different multidisciplinary research groups, one of the potential projects that we’re planning on is on looking at Finnish corruption, especially in public procurement in more detail and another one is relating to creation of innovations. So hopefully these projects will go forward and I’m also very anxiously waiting to hear what the European Commission is planning for the new Remedies Directive because that is something, the remedies I mean that I have always been interested at so probably I will look at that as well when we have more information on what are the changes to be made in future.


Well I think that’s a great way to finish the programme. Thank you very much Kirsi.  

Thank you Pedro.

#17 - Grith Skovgaard Olykke (Copenhagen Business School)

Abnormally low tenders in public procurement

Interview with Dr. Grith Skovgaard Olykke, Associate Professor at the Copenhagen Business School about abnormally low tenders in public procurement and mixed methods research (law and economics).



It’s great to finally have you here, and I’m very grateful that you accepted the invitation to be on the programme, and also to be one of the referees in the Early Career Research conference that we’re setting in for early March, and I was very happy with the kind of work that you did on selecting the participants.

Oh you’re welcome, and thanks for inviting me to participate.


Yeah, I’m looking forward to the conference, but we’ll leave that for the end of the show. Speaking about abnormally low tenders, this is your, I would say, your main area of research so far in your career, can you start by explaining us a little bit what are abnormally low tenders?

Well that’s a very good question to start with because there’s no clear answer, and that’s why it’s such an interesting research topic, I think. Basically there’s some agreement that there’s no definition of abnormally low tenders, which means that there’s a lot of discretion with authorities and with the national review bodies. Basically I think the idea is that the tenders are perceived to be abnormal though they don’t appear serious. But, then again, that’s a lot of discretion into defining what serious means. A general idea is also that there might be a risk that the contract might not be performed on the terms, and that could also indicate an abnormally low tender. The Courts of Justice has not been asked specifically what an abnormally low tender is, and hence my interest in this topic.


Yeah, I agree with you. One of the things that has always surprised me about abnormally low tenders, in addition to the fact that it’s not very well-defined, and the fact that is a very important limitation, a very important consideration in the whole area, what I find interesting as well is that even if a tender is found abnormally low, there is no obligation from the contracting authority to actually exclude such tender from the contract, it can still keep it.

Yeah but that was sort of the 2004/18 approach. Now with the new directive there’s actually an obligation to verify and to reject abnormally low tenders under certain circumstances. So that’s going to be very interesting to see what’s going to happen with the case law on this topic in the future.


Well, I hope it changes the practice because, for me, it never made much sense in the old directive that we would have a situation whereby a tender may be abnormally low but there would be no consequence for such tenders, so let’s keep an eye open for the near future on that area. In terms of the research that you’ve already done on abnormally low tenders, what have you found out that is important about this area?

Yeah well, to begin with, with my thesis, when I started, I was fascinated by this lack of a definition, so I set out to actually find out, with an EU law perspective, what is an abnormally low tender, and what I started out with was to think about where else do we have rules on low pricing. Obviously this would be competition law. What I did was to try to argue that a well-developed legal tool from competition law could probably be transferred into public procurement law, so I’m not talking about using competition law, I’m talking about utilising a tool developed in competition law to assess pricing in a procurement context. I call this the coverage of cost approach, because obviously competition law talks about you have to cover certain costs. So I tried to sort of transfer this into a procurement context. I think the approach is appropriate, in particular regarding arguments of coherence in the EU legal system, because if there is a difference between an abnormally low tender and a situation where a low price breaches competition law, there will be a risk of exploitation, of the public procurement context to distort competition and exclude competitors. So I think there needs to be coherence here, and that was one of my basic arguments.


In addition to that, and moving on a little bit to the work that you’re doing today, what has changed in the way that you are treating abnormally low tenders or the way that we should be treating abnormally low tenders?

I think so far we’ve had a situation where you couldn’t really do anything in these cases, unless the tender had been rejected. What I’m looking at the moment is Danish case law and Swedish case law on abnormally low tenders, and I see, a lot of these cases were the acceptance of low tenders has been challenged, but with no success because there’s no obligation to reject such tenders. I think it’s interesting to see what’s going to happen now with the obligation in the new directive where actually sometimes, having to reject abnormally low tenders when they’re low on certain grounds. That’s the Article 18(2)[?? 06:06] grounds, the social, developmental and labour law obligations that need to be adhered to. When you’re pricing a tender, obviously your price needs to reflect these costs . I think what we will be seeing, is going to be a massive change because now you’ve got a right to challenge the award to a low price tender, and this could be very interesting to see what’s actually going to happen. I think we’ll see a lot of case law and also think that the approach that I proposed in my thesis is actually supported by the new obligation to reject tenders, because as I mentioned, the tender has to reflect social and economic obligations. So, I think it’s a very interesting time for this topic.


What jurisdictions are you covering in the current project?

The current project is an interdisciplinary project, so I’m working together with Swedish economists. The law part is looking at what is the concept of an abnormally low tender in Denmark and Sweden, and there are differences I can already feel are there. And the economist is going to look at what kind of strategies might be identifiable in these cases, so what kind of strategic pricing might we see, and together then we’re going to look at how should strategic pricing that we see in these cases be handled legally, and also we’re going to try to say something about what’s going to change with the new provision and the obligation to reject.


On that note, do you think that there’s a relationship there between abnormally low tenders and what economists call the winner’s curse in auctions?

That’s not what we see.


It’s not what you see?

No, it’s not what I’ve seen in the cases that I have looked at so far. What I see in the case law is more the economic concept of unbalanced pricing, maybe?



So some prices high, some prices low, according to what you expect demand might be, and it’s all the prices are collected in the assessment of the price, then you could have a good score on the price, even though your tender is actually going to be more expensive for the contracting authority because you’ve higher priced some of the items, that’s going to be demanded a lot, or that’s going to be the main part of the contract, but since other price items weigh the same in the assessment, then you have a very good price and you score well on the price. So I think that’s what I’ve seen so far, but I’ve only just started to go through the Swedes’ case law now and there’s a lot of judgments . It’s also acknowledged by the Swedish court that we see strategic pricing and basically that it’s OK.


Well, that reminds me, it connected with something that we’ve not talked about, which is additional works. I mean that reminds me, when I was a lawyer, and this was before the directive 2004/18 was [?? 09:05 resposed] into Portugal, I remember being a procurement lawyer and having discussions with clients, both private and public, about the sustainability of certain bits, and pretty much everyone in the specific sector, which I’m not going to disclose, obviously, they always under-priced their bids at least 25% because the Portuguese law at the time allowed them to, during the contract performance, to find ways to claw back those 25% legally. Things changed a little bit and the Portuguese law now imposes a 5% limit, so I don’t know if that’s changed the practice in Portugal or not. Now, what may happen nowadays, which, if I recall correctly in the directive, there’s not a hard limit or very strong limitation on additional works as it was in the past, what you may see is that there will be a tendency for suppliers to under-price their offers, their bids, precisely because they think that once they get the contract, they may try to make it up in additional works in, or additional services.

Yeah, yeah, I agree. You are referring to Article 72 on amendment of contracts,? I agree, and I think maybe the obligation to reject abnormally low tenders sort of goes in the other direction, because at least you have to share that some of the specific costs are covered in the tender, otherwise it would not make sense having an obligation to reject. So it could go both ways I think, but you’re right that the discretion to change the contract after it’s been entered into, puts at risk that you have to pay extra for the same.


Yeah, I agree with you in that as well. So looking a little bit further down the line, into the regulation, what do you think that we should do regarding abnormally low tenders? Do you think that the decision by the lawmakers, including mandatory exclusions of abnormally low tenders in the current directive, is a good thing or we should move back to the old system?

Oh, I think it’s a very difficult question, and I’m not sure that I’ve actually made up my mind.


That’s fine.

I think we have to see what happens, you know, what kind of cases we’re going to get. I know that in the Swedish practice I’ve been looking at, this is very preliminary research results, we already have quite a lot of cases on hourly wage, you know, you should have a specific hourly wage in your tender, and that’s sort of, something that I think we’ve got to discuss in the future, and I don’t know quite how this is going to end. So I think I’ll sit back a bit and watch and see what happens before I make up my mind.


Let’s explore that a little bit because that is a topic that is creeping up here in the UK as well, and at least some, I would say, some guidance, and if I’m not wrong as well, at least in the Scottish position of the directive, which is an effort to put the floor on the wages, which might be different than the minimum wage here in the UK.



Is that what’s happening in Sweden as well?

Sweden’s a bit complex because they’ve got a “problematic” collective agreement system. They’ve been challenged with the Laval case, and they’ve made some changes. It’s the same here in Denmark, we don’t really know what the minimum wageis. So we’ve got some issues. I think what they have been doing is trying to sort of fix a minimum hourly wage but not necessarily with reference to collective agreements, and some of the tenderers arguments in the case law that I’ve read is that we are actually adhering to the collective agreement but the minimum wage set by the contract unfortunately was higher than this. But it’s not really been explored by the courts, so I think maybe it’s the same trend we’re seeing in the UK, but there’s not been any arguments regarding the posting of workers directive, etc, because it’s been national cases.


Yeah, OK, makes sense. Moving onto the second topic that I would like for us to focus on today, which is your experience with interdisciplinary research. So as I said at the start, you have a mixed law and economics background, I think that it’s a very interesting mix of skills and approaches. What has been so far your experience in trying to bring those two different, let’s say, frameworks of thought into research?

Yeah, I guess I’m a bit biased because, as you said, I’ve got this business administration and commercial law education, and I’ve got difficulties letting economics, leaving economics completely out of my mind-set, not that that’s a necessarily good thing to do. So, I’m always thinking economics when I’m doing my research, and what I’ve tried to do recently is to team up with economists to do interdisciplinary research. I’ve focused mainly on law myself, but I’ve got some ideas of the economics that could be interesting to do as well, and then I’ve teamed up with an economist to do that part, and that’s been really fruitful and interesting for me.


So why do you prefer to bring on board an economist instead of doing it yourself, the economist’s bits, so that you can focus on law? Why do you do that? What is the benefit?

Well the benefit for me is that I’ve majored in the law since I finished education in 2006, so my economics is a bit rusty. I’ve got some of the general ideas and I just want to make sure that if I do research then the economics should be an appropriate level as well.


Yeah, that makes sense. I mean, that’s one of the things I’ve noticed which is there’s only so much you can do coming in from a different discipline.



It’s good if you are able to bring, let’s say, some tools off a different discipline into our own. So for example, for my PhD I used social sciences research methods, and certainly I would like to use quantitative research methods in the future, with my research. But at the end of the day I’m also very aware of my own limitations.

Yeah, exactly.


And mine are even more pressing than yours because you have an education, an economics background, you have an education in economics, I don’t, so I left maths and all that section of knowledge in my education when I was 15 or 16.



So I think that is a very important shortcoming in my view about legal education in general, is that it does not foster in the students the development of those quantitative skills, and that’s something that, for example, I would like to work for in the future as well. But going back to the research, as time goes on, one of the things I’ve noticed is that, especially for us lawyers or legal minds, it’s very hard to have, I wouldn’t say complete but a realistic view about the problem if all the skillsets that are brought to bear in that research, are legal skills.

Yeah, I agree with you, I think it’s very important to have alternative disciplines or other disciplines having a look at the problem as well. Sometimes it’s completely new perspectives and I think it’s very important to actually get one step further, or, not just one more, but some steps further by bringing in another discipline. A very good example of this is a paper that has just been published in the European Journal of Law and Economics, which I co-authored with an economics professor on the concept of services of general economic interest, because it was something that I’ve been wondering about for a while, and I know that different people have tried to say something about, what are the services of general economic interests, from a legal perspective. So I talked to this professor, whom I taught a course with, that also concerned services of general economic interest, and we decided to try to work out what would be the economic point of view on this concept, and we ended up doing a very interesting article, I think, I had good fun doing the research, and we had great discussions on how we could bring economics into this concept which was mainly legal.


I agree with you because in my own research, what I’ve seen, and this is, as I said, an idea that has developed, that I’ve developed as time goes on, is that in the end, I mean, the legal perspective, it’s a means to an end, and that’s something that to me, as lawyers, sometimes miss out, we think that law is an end in itself. So when you’re drafting the laws, you’re thinking about the legal implications and not the wider implications of the law, and when we’re interpreting it, we’re just looking at the legal issues and the legal consequences instead of looking at actually what are the real consequences of whatever legal interpretation we are suggesting.

Yeah, we’re limited in that way, aren’t we?


We are, we are, but we’re so full of ourselves [?? 18:03] think, no no, what is relevant here is the legal solution, is the legal issue, and I said, no, I don’t say that’s the truth, the truth is, yes, there may be a legal issue and there may be a legal solution, but they are means to an end.

To solve this question.


Yeah, if we just bring the legal frameset into discussion, we are limiting ourselves in the way that we can actually make any changes and make an interpretation that is not only, let’s say, legally irrelevant but also relevant and consistent with what is out there in the world.

Yeah, yeah, exactly. And I think there’s a problem with the law in general because the law is sort of a democratic outcome, you might say. Sometimes you spot that there are inconsistencies and different incentives combined in the same legislation, which doesn’t make sense at all. And we can use economics as well to look at the inconsistency.


I find that interesting as well. So one of the things I say, or I did say when I was teaching procurement law to students was if you look into the old [?? 19:26] services, you can easily find out who are the strongest lobbies in Brussels.



And I’ve always been amazed, for example, with the exclusion of legal services, I mean, there is no actual difference, in the end, that would justify the special treatment.

Yeah, yeah, true.


So I find that interesting, so that is more of a criticism to our shared background in law than anything else.

I think it’s more politics than law, isn’t it?


Yeah it is, it is more politics than law, but the thing is, again, it’s more politics than law but it ends up in law.

Yes, exactly.


So, to finish it off, what have you been able to get from having economists working with you in your research that you have not been able to get by yourself, even though you have an economics background?

I think it’s the insurance that economic methodology is correct, you know, there are experts on the topic, and you can choose an economist for what you need in your research, OK, because they’re so specialised on their specific theories and methodologies. So I think by bringing economists in you get an expert on a specific methodology or theory that you couldn’t bring in yourself, even though you have some ideas about economics and you’ve read all the general theories. And I think that’s very important. And they’ve also got, the people that I’ve been working with, some skills in, for example, handling of data that I’ve not been exposed to during my education. I think, they are specialists as well as we are, you know, so I think it’s appropriate to acknowledge this and bring economists in rather than just feeling that we can do it all ourselves by sort of general assumptions or reference to general theories of economics. I feel limited, you know, in economics because I focus so much on law. Even though, maybe I could study or maybe I could read up on things but I would be sure to miss something, so I’d rather bring in an expert from another discipline.


Yeah, I think that’s a very sensible approach. One of the things I’ve noticed as well is there’s a huge gulf between our language, the legal language, also the economist’s language, so sometimes it’s difficult to bridge that gap, and looking at your CV, the kind of work that you’ve done with economics, and obviously perhaps because you’ve had an economics background, it’s clear to me that you are able to communicate and bring together and draw out the knowledge from economics by using, [?? 22:04] from economics but also being able to develop your own ideas, you know, research by having good conversations with them.

Yeah, yeah, true. That’s one of the advantages of my background, I think, because I know what they’re talking about and I think it’s interesting to actually try to bridge those different terminologies between law and economics and just sort of an anecdote, the first job I had was actually as an economist in a consultancy house, and I was brought in in order to bridge the terminological or linguistic gap to the lawyers, you know. So that is a real issue where I think you have to be really open and ready to discuss these issues, but be open to the other approach from economics and other disciplines.


Yeah, I think that’s, again, a very sensible approach, and a great way to end the interview. So, thank you very much Grith.

You’re welcome.

#16 - Guilherme Lichand (Harvard University)

Can a  little bit of corruption be good for your health?


Guilherme Lichand's Harvard page and his job market paper.



It’s great to have you here Guilherme. Your paper it’s fascinating and thought provoking. Why did you decide to look into the issue of corruption in the health sector and its potential downstream effects?

Great. The idea – not only in health, but more broadly – is that while we have increasing evidence of what tools can most effectively fight corruption (in particular, audits have been shown to significantly decrease missing government expenditures), we have much less credible evidence about what is the effect of those tools on the outcomes that corruption is supposed to detrimentally affect. The distinction matters, because bureaucratic performance has multiple dimensions; rent capture is just one of them. On the other dimensions we have, for instance, the quantity and the quality of public goods’ provision. So while, on the one hand, limiting the extent to which bureaucrats can capture rents should mechanically increase resources flowing towards public goods’ provision, on the other hand it might decrease incentives for bureaucrats to exert effort on those other dimensions. Theory predicts that the implications of deterring corruption on public service delivery should be ambiguous, and, hence, we try to understand whether deterring corruption helps or hurts public service delivery. That’s the prime motivation on the paper.


Any reason why you picked Brazil over any other country that you could have done this research on?


Brazil introduced an anti-corruption program nationally in 2003, so it provided a great opportunity to try to answer this question. First, because the programme was introduced right at the mid-point of the Mayor’s political term. The term is from 2001 through 2004, and the programme was announced exactly in January 2003. Second, because these audits are retrospective. Basically, when auditors go there, they follow the paper trail of every investigation, at least up to three years prior to the time of the audit. This way we have the data we needed to try to answer this question. That also takes us to the point of why health. The answer is, first – given the audit data that we needed to code – because we needed to restrict that attention to a particular area of public policies, because it is a huge amount of data. Second, because of outcomes – availability is very good at the municipal level for health. We have a lot of indicators at the municipal level that are measured for all municipalities every year, and that was really what we needed in terms of data requirements to try to answer this question.


What's the main finding of your paper?

On the one hand, we find that the program greatly reduced corruption – and by this I mean procurement manipulation, off-the-record settlements with vendors, and resource diversion. These kind of problems, according to the audit reports, are greatly decreased by the introduction of the program. But on the other hand, what happened is that the program also dramatically deteriorated the quality of health indicators – and by this I mean hospital beds per thousand inhabitants, immunisation coverage, or household access to piped water and proper sanitation, in comparison to other health indicators that involve much less procurement. So to anticipate the results, what basically happened is that, it seems, we thrown the baby out with the dirty bath water. The programme was a major institutional improvement in terms of transparency, but by focusing so much on corruption – both in terms of public opinion and administrative punishments –, what seems to have happened is that spending fell by so much that corruption per dollar spent has actually increased after the program.





So let’s break that down into sub-questions or comments. So the audit programme did reduce corruption so you guys were able to measure that, right, okay, so that is one element. So in terms of the let’s say, first degree objective of the programme it worked as it was supposed to work in terms of reducing corruption at least in absolute levels if not, as you were saying, in relative levels per spend.

So when we look at the percentage of investigations that were coded as corruption, looking at these audit reports, indeed it is the case that the baseline incidence of corruption is decreased by half. It is a major effect. Just to give you a sense, among those transfers that involved a lot of procurement – we look at how programs earmarked to specific spending, like buying hospital beds or contracting hospital reforms, those sort of transfers –, before the program, about 37% of our investigations were coded as corruption, some sort of the problems that I previously mentioned. After the program, that rate falls to about 12%, and this is the same rate for those audit transfers that did not involve procurement at all – like those earmarked to pay wages of doctors or local procurement staff. So, basically, the idea is that, as you said, if you just focus on corruption, then the program would do exactly what it was supposed to do: every investigation now had much less of this procurement problems that were very prevalent beforehand.


But on the other hand?

Yes, on the other hand... before we even tried to look at the mechanism, if we look at the outcomes – because, you know, corruption fell by so much you might think that health indicators would become much, much better after this –, what we find is that these health indicators (comparing again these two sets of transfers: those with a lot procurement and those basically with no procurement) become significantly worse. Just to give you a sense: in a cross-sectional comparison, the magnitude of the effect is equivalent to losing between half and all of the support of the federal government towards municipalities’ health budget. Because municipalities depend so essentially on federal redistribution to be able to provide those local public goods, this effect might mean that they have no budget left, and this is consistent to the effects when we look at spending. After we found these effects on the health indicators, we wanted to understand the mechanism. What was happening? Why was corruption going down but health indicators becoming worse? So we looked at two other dimensions of bureaucratic performance. First, quantity – proxied by spending. What we find is that using the same strategy, differential spending across the two sets of transfers fell by at least 50% after introduction on the program. This effect is consistent with the deterioration of the health indicators. And just by residual, since corruption fell by about 30 percentage points (the percentage of investigation coded as corruption), and since spending fell by at least 50%, this must mean that corruption per dollar spent must have increased by at least 20 percentage points. So that’s where this figure comes from.

We also look at quality: from these audit reports, we can also look at implementation problems. We find that, basically, infrastructure problems abound, infrastructure is no longer functional (or high quality), and also medication stock problems abound after the program. Because you’re not spending, then all these problems are coming about and this is consistent with the overall story that (i) either because bureaucrats can no longer capture rents they have less incentive to exert effort in these other dimensions or, (ii) and this is something we might talk more about, these procurement guidelines are so complex and the quality of these local procurement staff is so low that it’s now easier just not to spend and blame the federal government for not transferring resources than perhaps running into a procurement mistake and being labelled as corrupt.


We’ll address that in a minute. There was something that I was listening to you and I think it’s important to probe a little bit further there as well, which is it makes sense that the outputs or at least a proxy for the outputs, would go down if spend goes down as your analysis show so, you know, fewer beds per a thousand inhabitants, worse outcomes in terms of immunisation or the availability of medicines, all that makes sense because you’re talking about the health budget and how public procurement is really in the health system work. What I didn’t understand was the relationship with the access to piped water and as well to the waste treatment so how do you connect those two elements? Because if I get it correctly, is the audit then to the whole municipality or just to the Health Department?

So these audits look at overall expenses, it’s just that within health and education they look at the whole set of transfers whereas in other areas they look at a sample of the transfers. But the thing is, there are some health programmes (in Brazil at least) that are targeted to improving the connection to piped water, the quality of water and sanitation, and so when we choose indicators we look at what indicators the Health Ministry uses for monitoring and evaluation.


No, that makes sense now, it’s just that I wasn’t seeing the connection right immediately because the other ones are obvious and that one I knew that there must be a reason there, I couldn’t just find it. In terms of your analysis, you’ve done something that I find very interesting, is that you decide to separate the analysis in accordance with different procurement intensity so you’ve looked both at procurement intensive processes and procurements not intensive processes if it makes sense, why did you do that?

So you know the program involves random audits. So one might think that you could just look at the effect of the audits themselves on corruption and on these health outcomes to understand whether the mechanism we propose makes sense. Now, one problem with this is that, although we can indeed do exactly what I just said, most of the effects of procurement might perhaps have happened at the announcement of the program, even before any audits actually took place. So to understand the overall effect of the program we have to resort to a differences in differences strategy. We can compare (because we have the retrospective data), municipalities before and after both the Mayor and the local procurement staff learned they could be audited – so that’s one dimension of comparison. But if we just did these over time comparisons, many other things changed in 2003, so this is not enough to really pin down the effect of the program. So the idea is to look within municipalities, to different sets of transfers: those that involve a lot of procurement, and those that do not. The idea is that, first, the program should have had a differential effect across these transfers. Because most of corruption is linked to procurement problems, it should have had a higher effect on those transfers that involve a lot of procurement. And second, any differential in corruption across these two sets of transfers should have remained constant otherwise. That’s the assumption for relying on this strategy to try to pin down the effects of the program. We also use recent nearby audits as a complementary strategy (because they are random) for trying to assess the same mechanism. Roughly, the effects that we find are consistent. So both the overall effect of the program and those of audits themselves are in the direction of decreasing corruption, but also of decreasing spending by so much the health indictors actually become worse.


Yes, okay. Speaking about the audits, are they really random, i.e., is really a lottery or is this that the official claim from the system for the outside world is internally they pick who they want to do the audit to?

That’s a good question. This program is a joint venture with the National Lottery, so the draws are in television. Every two months or so (at least in principle), about sixty municipalities were supposed to be drawn. The first papers to exploit variation from this programme were by Claudio Ferraz (PUC-Rio) and Frederico Finan (Stanford). They try to understand if these audits are indeed random, so they look at a broad range of variables, political parties included, and they don’t find significant differences. Each municipality has a 1% probability of being drawn, they do it in this way every draw. What we had to do involved an extra step: to show that not only audits are random, but also that conditionally on municipality’s characteristics, recent audits that happened nearby are also random, so that we can exploit not only if a municipality itself is audited, but also if a neighbour is audited, it this raises the perceived probability of you being audited in the future – out of a behavioural mechanism – to look at the effects of interest.


Yes, the reason I was asking this is because in Portugal theoretically you are selected randomly to provide evidence about your tax situation and it was claimed originally to be a lottery but I mean the same people were getting the letters from the IRS, the Portuguese IRS, every year so it was very clear quite quickly that there was no lottery involved. What I did not know is that in Brazil actually it’s really a lottery and it’s done at least apparently above board.


Right. So there are many oversight programs in place, some of which are not based on random audits. The state tribunals do a bunch of audits which are motivated by either some event or even by a private investigation, so it’s an overlay of different institutions. This program specifically is based on random audits, and that’s what makes it nice at least for the investigation in this paper.


Yes, it makes perfect sense because if it’s random it’s one less variable or one less bias slash problem you may have with the data that could be influencing you.



Very well, moving onto another topic which is how transferable is this finding or these findings to other areas of public sector that are also subject to procurement analysis of expenditure, particularly areas that are very procurement intensive like say, for example, public works which is always usually the one that comes to mind? 

Our thought is that there’s nothing really specific, it seems, about this program in Brazil that would prevent the findings from being extended elsewhere. First, this kind of bureaucratic politics through which the federal government transfers resources that would fund the provision of local public goods, with a local bureaucracy which is responsible for running procurement before these resources can reach their final users – citizens – in the form of these public services, is the modal workings of public bureaucracy in the developing world, it’s seems to us. And there is evidence from other settings that are very consistent with this findings. There is a paper, from 2015, in the American Economic Review, for instance, about a program in Nigeria, showing that when it monitored bureaucrats on several dimensions of bureaucratic performance then they exert less effort in those dimensions; they try to divert efforts to dimensions that are less subject to monitoring. This is consistent with this idea that whenever you have this layer – bureaucrats that must procure resources before they can actually reach citizens –, you are subject to this kind of tension between actually procuring / spending, on the one hand, and on the other hand, making proper use of public funds. So the only difference that I can foresee is when, in some settings, resources are directly transferred to the end users. There is a paper, from 2004, with data from Uganda, in which they tried to investigate the effects of a newspaper campaign aimed at ensuring that the federal resources would reach schools. They claimed that, in Uganda, before the campaign, basically the modal transfer to schools was zero. So most schools were getting basically nothing out of what they should actually get. After the newspaper campaign, then citizens put pressure on the federal government, these resources are really going to the schools and then educational outcomes improve. But there you see the difference: there is this layer of bureaucrats having to procure which is not present, so I think that explains the difference, in my opinion. In any case, wherever you have bureaucrats, the same logic should apply.


It’s very interesting for two reasons, the first one is that effectively its principal agent relationship issue that is at hand and one of the things that I have noticed that is mentioned on your paper is that because taxes are collected centrally but spent locally that there is less emphasis or less say pressure from the actual supposed beneficiaries and taxpayers in that local area to actually make sure that the money is well spent, which is the difference with Uganda, it’s a big difference.  

That’s right. I mean, in Uganda, perhaps, in this particular sense that it’s not raised locally so that citizens have a lesser incentive to pay attention to it, I think this part is exactly the same. It is just that when you put it on the newspaper, then citizens have a higher incentive to monitor. There, this makes resources get to the schools; and that is where they should get at the end, so you’ve already done the job. In Brazil, there are other papers that show that when you put these evidence of corruption in the newspapers, citizens punish politicians. There is a paper showing that the probability of re-election falls by about 30% after – comparing two municipalities that are equally corrupt, but one was exposed just by randomness before the election, and the other after it. So citizens do take this information into account in both cases. It’s just that, in the Brazilian case, this is not enough. In Brazil, the only way a bureaucrat can lose their job and go to jail or have to pay a fine out of their own pocket is through this program. So you can imagine how high the stakes are for making a mistake or actually capturing rents whereas the upside of doing your job right and making sure the resources are properly spent, currently, is very low. The rewards from doing a good job as local procurement staff are very low, there are no incentives for public service delivery but there are very high punishments for corruption now.


That’s very interesting and it makes perfect sense. Another point I picked up  was that you were talking about, from a developing world perspective, and saying “oh this is how we organise let’s say the health sector or the way that procurement is done in the development sector”. Having said that, I mean the same thing happens in developing nations, I mean, for example, here in the UK, thinking about the health sector in England you have a number of different authorities, Health Authorities, which are called Health Boards, each one responsible for one, two, three hospitals. In Wales you can have the same structure with different people. In Scotland as well, in Northern Ireland the same as well in an even smaller number and in the countries that I know, Portugal, Spain and to a certain extent France and Germany, there is also a big effort to try and put the decision, the decision making even in procurement as low down the chain as possible and as close to the citizens as possible as well. So, for example, in Spain you’ve got regions as you have the states in Brazil, and the regions have or are responsible for their own health sectors. So I think that if your findings are indeed transferable they can be transferable not only in a developing world setting but also on a developed world setting as well.

I don’t have much to say about this; what I can say is that what we find is consistent with an old discussion from the 1960s. For instance, Sam Huntington said that during the process of development of the US and the UK, corruption was one of the main drivers of development. Professor Steve Kelman, from the Harvard Kennedy School, very often in the media emphasises that perhaps we might be over-monitoring – paying too much attention to complying with too rigid procurement guidelines, versus you know, paying attention more broadly to the quality of public services. So, you know, from the paper, there is a limit to what we can learn about other settings, but I think it is consistent with the more general discussion here.


Yes, picking on that and moving to the next question, one of the things that came to mind as I was reading your paper is that the fact that the bureaucrats, the civil servants, know that they are being monitored or can be monitored and can be audited and hence the change of practice and corruption goes on but also the delivery goes down as well with it, it maybe that even if those agents are not corrupt by themselves the fact that they know they are subject to more scrutiny and more accountability actually raises the stakes for them so if they can avoid doing a procurement process that is deemed risky or could be risky, as you said, it’s a question about incentives, if they don’t have the incentive to provide a good service they might as well avoid the risk of getting the fallout from anything going wrong even if it’s not, you know, directly corruption.


I think you may be perfectly right. We cannot fully attribute the results to the fact that local procurement staff can no longer capture rents to the same extent, such that there’s just less incentives to do everything else. In our paper, we are not able to separate between the two stories. Indeed, from speaking to both auditors and local bureaucrats, these procurement guidelines are so complex and, as I said, the quality of the local procurement staff is so low, in comparison, that it’s just easier not to spend and put the blame somewhere else for the lack of delivery than to run this risk of being mislabelled as corrupt from running into a procurement mistake. So the general message is that not only perhaps we should refine how to balance these incentives, as I’ve said before, but also try to raise the capacity of these local procurement staff. With any capacity building program, we shall see have positive effects on both spending and corruption, if it decreases the likelihood of these procurement mistakes. And also, perhaps, we could redesign the program to focus on larger transfers, because if you think about the rational decision to capture rents, the incentives are really there regardless of the probability of being caught for the really large transfers. And these are the ones for which the scare effect on spending from the risk of being audited is not going to matter much. Basically, if you think what the program is doing... if there is this component of being afraid of running into a procurement mistake that is scarring bureaucrats out of very small transfers – for which they probably wouldn’t have been corrupt anyway and, even if they were, the consequence of that for the public budget would be very small. So there is a question there of the actual design of the programme and also of capacity building at local level that I think these findings raise.


Yes, I think those issues are raised and raised a lot at the end of the paper and it’s certainly something that echoes what has been argued here in Europe and you think that in Europe procurement is very good and full of, people full of capacity, that is not the case. Even here in the UK there’s a huge distribution in terms of the quality of the contracting authorities and also the quality of the people that actually deal with procurement on a day-to-day basis and in fact here in the UK with the budget cuts over the last five years or so,  procurement capacity has probably gone down in most places because they’ve lost staff, senior staff, and they haven’t been able to replace that senior staff. So it’s very interesting because when people ask me “Where should they save money?” I say, “Well you should save money in procurement certainly”, but probably the best way in the long run to do it is to actually up-skill your staff to make sure your staff is competent and can deal with complex procurement guidelines which are always in Europe as probably as complex as the ones in Brazil. So it’s not only a problem in Brazil I would say.

Right, it makes sense. It always comes to mind to me, it’s not only about having higher-skilled people in your staff, but also about having models for basic projects and everything. For instance, one phenomenon we see there, just anecdotally (and I’d like to document this), is that when a capital city procures a complex public work, like a school or a big piece of urban infrastructure, a lot of the other cities copy, procure the same.



Because they don’t really have the skill to write the terms of reference. This is a very complex thing. So this just comes to show that you could do more. Of course it comes out of not enough skill from these local procurement staff in these other municipalities to do the same on their own. But, also, if either the state level or the federal level provided these basic projects as a public good, then they could use as a basis for what they actually did. That’s also an easier way, and you have economies of scale in doing this. Of course, at the local level you cannot do it but that would be a huge contribution that the federal level, for instance, might do besides just trying to do small training programmes at the municipality level.


Well, speaking anecdotally as well here in Europe the same thing happens which is I talk with procurement officers very regularly and they will say to me, “Oh yes, we’ve been using this, let’s say, this template that we got from our colleagues in a Council down the road”, and I just say, “Okay, so what quality control have you done on that?” “They’ve used it so I suppose it’s good”, yeah, right, okay, so.

So imagine if the federal government provided this template? That could be a huge contribution.


Yes, I agree with you, it’s a good idea, I never thought about that in that sense, I mean there’s been some attempts to try to do stuff like that or a little bit like that but very much on an anatomised basis and not on a very centralised and very controlled basis which I think would be necessary. Another thing that I think it’s relevant for the discussion, some colleagues of mine who are in Wales at my previous employer, Bangor University, they looked into the procurement practice of three small Local Authorities, so three small municipalities here in Wales and they found 600 people with buying power so with the capacity to buy stuff and obviously they were subject to rules, some of them are internal to each municipality, some of them are national and some of them come from you know from the, what we call the premium directives but I just ask myself how will three small Local Authorities be able to up-skill all these people to make sure that they are good enough, they just, even if bringing into the level that they’re just average procurers. Or is it possible, it’s pretty much impossible and the only way I can see it, is that if you actually reduce the number of people that are involved in these processes and you increase to a certain extent the transparency and the monitoring to make sure that you are actually capturing the information.  

Right, that almost comes down to the organisational aspect of it.



It might involve a lot of people, because these municipality in Brazil might be very complex, with up to 20 different secretariats. And that makes sense, because they’re responsible for so many local services. But it’s true that it’s impossible that you would think that each of these secretariats are going to be equipped with the best procurement staff, so you should expect that perhaps the secretariat of planning might centralise all the best staff in procurement and everyone has to run procurement through them. But it comes down to systems, and retention, etc. And, you know, the discussion is usually not there.


I agree, I agree, I fully agree with you. Well our half an hour is over so I’d like to thank you for this very enjoyable conversation we had, it was great. As I said you’re the first one from economics that we’ve had here in the podcast and I hope you’re not the last one before we finish the series.

Great, my pleasure.


Thank you very much for coming Guilherme.

Thank you. 

#14 - Richard Craven (University of Leicester)

Research methods in public procurement (law)


Interview with Dr. Richard Craven, Lecturer at the University of Leicester.His research interests lie at the crossroads between law and social sciences, with a particular emphasis in public procurement research methods. Today’s talk will have more of a shared approach to it than usual as we both have something to say about research methods in public procurement.


It’s great to have you here, we’ve known each other for a few years but I think it’s the first time that we are getting back together and talking about procurement and procurement research methods.


Let’s start with your PhD, what research did you do for your PhD and what methods did you use?

Okay so my PhD completed in 2012, looked at the legal framework in the UK for regulating public private partnerships, so what we know as competitive dialogue, and then as you said, it wasn’t just a case of looking at the law in books. So looking at case law, looking at what the legislation said about how people are supposed to behave, it was more we went out into practice and conducted interviews with lawyers interpreting and applying the rules in practice, policy makers and procurement officers and contracting authorities, so I suppose just to see what sort of legal uncertainties were being interpreted in practice and how difficult areas may be where we might not want to comply with the rules, how that was approached in practice and, yes so really just that sort of, yes law in books to law in action-type approach.

I mean that’s very similar to the research I’ve done and I did in 2010, that I finished in 2010 for my PhD. I also was looking forward to see how competitive dialogue had been implemented and was being used in practice in Portugal and Spain. What I find very interesting is that it’s very different from watching or seeing the law being deployed in action by agents that are actually using and sometimes abusing the law in comparison with what you can read in the books, what you can read let’s say in the directives or in the laws and regulations.

Yes same here really, I think probably in the UK I think there was a lot of say reporting in the trade press that competitive dialogue was causing problems in practice, but I think it was about seeing overall, why were those problems being caused, how was it being interpreted maybe to cause those problems and what strategies were adopted to get around it. So yes, it was fascinating really coming from sort of a law background, you know you’re always studying law as in, ‘Or what are these legal rules?” and then sort of taking a completely different perspective that actually the law is what people are doing in practice, not necessarily what it says in the books yes.

Yes, I think that’s a very good point to remember, back in 2004, 2005 let’s say or 2006 maybe, the commission say in interpretive communication or whatever about competitive dialogues using some made-up examples of where the competitive dialogue could be used for example if [?? 3.25] authority wasn’t sure to use a bridge or a tunnel to cross a river, those kind of examples which made perhaps perfect sense in theory and for the people that were involved in drafting first the Directive 2004/18 and then obviously that communication or interpretation. Then in practice there was no correspondence with reality I mean, I have never seen an example where that kind of problem actually happened in practice.

Yes definitely, I agree, and hopefully then I suppose that’s what our PhDs contributed to good policy-making, because we’ve seen the law reformed now to something that’s maybe a bit more commercially sensible. I think originally without those commission interpretations, the law wasn’t being interpreted in sort of a way that was relevant for practice, and it was because of that sort of research, and we weren’t the only ones, but sort of highlighting that sort of miss-match between the, yes you know maybe for transparency reasons we might like to have these rules, but for commercially sensible reasons it’s just not going to work necessarily. So yes I completely agree there.

Well that takes me to the second question which is, what can we say that is wrong or limited with purely black letter or doctrinal analysis?

Well I don’t want to be too critical of doctrinal analysis, you know I enjoy reading doctrinal analysis sort of, obviously it sets up my work because I do empirical research, but I think it’s always that it can only take us so far, and particularly with public procurement in the UK as well where we have limited case law, we have legislation that’s uncertain in areas and complicated, and also a lot of guidance documents, so what we might call soft law so you don’t have to comply with it but you know, is authoritative in some sense. So a black letter legal analysis can, yes it can tell you what the law might be but actually it can’t tell you what the reality of the law is, what’s actually happening out there so yes, it can only take it so far really.

Yes but I think that is a very important point to make in a very important limitation to highlight, which is it only takes us this far but then it doesn’t allow us to actually know how the law is being used in practice and how to improve it, because at the end of the day to a certain extent, like with the analysis, it’s a little more than just opinion yes, it’s a legal opinion, but it’s only that.

Yes, I can only agree yes, it’s just one side of it and you know, there’s sort of great academic debates that go in say public procurement, public law but in practice just isn’t happening in line with that so it’s, as I say, just limited value isn’t it.

Yes, personally I’m always much more interested perhaps because I used to be a lawyer because I became an academic, I’m always more interested in looking at what is the job to be done that a certain piece of legislation is trying to solve and see if it’s actually solving it in practice, looking at in reality what it is doing instead of what people wanted it to do from the outside.

Definitely, and I suppose it’s in the UK, I suppose it’s just a modern way of doing things as well, so we’ve always, sort of legal researchers have always favoured doctrinal analysis and it’s only more recently that we’re sort of embracing social science methods which can only be a good thing.

Well to be honest, I mean it’s much easier and to a certain extent cheaper as well and less time-consuming to be just black letter analysis instead of doing empirical research.

Yes definitely, but I mean I have colleagues complain that you know, with doctrinal analysis they can’t apply for funding and what have you whereas actually empirical researchers, we can always make case for funding which I think is always viewed quite positively.

Yes that’s true, I hear the same thing both here in Swansea where I am now and also at Bangor University where I was before, the colleagues with the more traditional skillset in terms of legal academia, they find it very difficult in this day and age to find funding.

I think it just shows you what kind of you know, policy-makers, they want that, they want to know what’s happening out there, they don’t just want to know the particular sort of view point or analysis of one person necessarily.

Yes, agree with you. Now moving onto another question, why should we be using mixed methods for legal research, what are the pros and cons of doing it?

I suppose, I mean we’ve outlined the pros to a certain extent.

Well some of them at least.

Yes, and I suppose some of the cons as well in terms of the, sort of the expense and the time, I think also without wishing to be critical of black letter analysis [?? 8.15] or overly critical, I think in the same way black letter legal analysis can be, empirical research can be done very badly.

Yes that’s true.

So I think it’s only good if it’s done sort of quite robustly you know, you’ve got strong methodology there, and I think if it’s not done that, it’s very easy to pull apart and it’s just time wasted, so you know I don’t think it’s something people should just, or researchers should just jump into, I think it is quite hard to do and it needs to be done correctly. But the pros there I mean, as we’ve said in terms of what it tells you about the law and what is happening out there, and the way it can inform policy and show where legal rules might not be achieving what they were intended to achieve, I don’t think there’s any, well the alternative being black letter would not be a suitable approach for that.

I think you’ve made a very good point there about the difficulty of doing empirical research well. That is one of the limitations that we have in legal academia, is that by and large in the undergrad studies and even at the postgrad level, by and large we do not foster in the students the development of research skills other than the traditional research skills.


And I think that is a very important constraint that we have then further down the pipeline when people start doing their own research in their PhD or even after the PhDs when they become early career academics, it becomes very hard to retrain and regain those skills that were not gained perhaps in the moment they should have been gained.


I notice that for example, not exactly as much with empirical research but certainly with more economics-minded research, when I try to blend law and economics I can easily get to the limits of my knowledge and the limits of my ability of doing that kind of mixed-method research which I’m really interested in doing, but because I do not have the right background in terms of quantitative skills it becomes really really hard to do it well.

Mmm, I think there’s a lot to be said there, but at the same time though I mean, I think empirical research is never going to be perfect is it, it’s just about sort of justifying what you’ve done, there’s always, if you don’t write the findings of particular empirical findings I’m sure you can find some faults in the methodology somewhere and pull it apart, but I suppose that’s just the nature of it. So it’s never going to be perfect but it does need to have that sort of, that strength behind it as the sort of solid foundations and, yes I think the more academics are trained in it and I think universities do provide that sort of training now more and more, so they’re better.

On that note what would you like to have known at the beginning of your PhD that you know now about research methods, especially empirical research methods?

I think it’s probably that, that it’s not going to be perfect and things are going to go wrong, you know maybe interviews that are arranged are going to fall through and, or you know, the interviewees aren’t going to say the sort of things that you expect them to say or maybe you want them to say if it’s going to fit with a hypothesis.

Did you have problems with that?

I didn’t but I think you’re just worrying the whole way through because it feels so messy you know it doesn’t feel, you know I think doctrinal research, I think it often feels like you’re in control whereas empirical research doesn’t always feel that way because it’s so dependent, you know so sort of spanning over maybe a couple of years and it’s so, yes just hoping that people contribute and participate in your research but you know, I’ve sort of read around you know afterwards about other sort of empirical studies or famous empirical studies in different areas, and that’s really what they talk about. So you know, you have this book at the end that sort of details this amazing research but actually, getting those findings it was not an easy process and it did feel messy at times, so it would have been good to know that, would have been reassuring to know that.

I can certainly relate to that experience, I remember effectively discovering the main finding of my empirical research literally at the dying stage of the research stage when I was doing my final interviews, and it was actually my last interview in person that I did, I still had a few other ones to do over Skype or phone or something like that, but I remember going into that interview and still thinking “I still need something, there’s something here that I’m not getting”. And I had just had that eureka moment after the final interview, the final in-person interview.

Yes, I think any empirical research will sort of relate with that, and I suppose another thing that I would do differently is just keep on top of all my data a bit better, I think as I was going through I was sort of taking these interviews down and wasn’t sort of transcribing them straightaway and that, you know.


Yes, well occasionally and I think it is important to sort of just interview, transcribe and let that feed into the next interview.

Yes that’s actually one of the things I learned to do right from the start, which was to transcribe the interview as soon as possible after the interview has taken place where everything is still fresh.


I never had any technical problems but if you try to transcribe the interview much later after it was done and you find out for example whatever technology, piece of technology you were using wasn’t working in that scenario, you’re pretty much screwed.

You are yes, I was quite fortunate though, it was all on a digital recorder and, thank goodness, lesson learned.

Yes, what about let’s say other mixed methods of research like economics for example, have you ever been attracted to that?

No not really just because I suppose it just hasn’t been something that I’ve really sort of encountered in my studies, so it’s just been a sort of natural progression to get where I am, and I see myself more as a qualitative researcher, I think I’m more interested in people you know and how people respond to legal rules, so that’s what, sort of just regulation more broadly. So I think just following my interests, it’s got me to there but we still sort of go off an sort of continued the empirical line of things, so rather than law and economics we’ve been recently researching into litigation behaviour. So for example why case law and public procurement are so limited in the UK and that was more sort of questionnaires and numbers involved with also, interviews.

What did you find on that one?

A number of findings actually so looking at behaviour, we’re looking at sort of numbers. I think an overall finding that was limited case law because actually many suppliers didn’t see any reason for litigating, but actually they were happy with the process which actually was quite an interesting finding and you’re you know, maybe not feeling a need to pursue a challenge where there was only a slight technical breach, but then a long with that just the usual sort of reasons for not bringing a charge, be it sort of cost of litigation and fear of upsetting, you know biting the hand that feeds, upsetting the public sector who you might be working with, you might be bidding for further contracts from.

The famous fear of blacklisting?


In a sense, that doesn’t seem very different from the research that Despina [?? 16.04] did maybe fifteen years ago or twelve years ago.

Yes, yes, so it was sort of building on her research really, but it was sort of a modern, you know Despina’s research was sort of pre- I suppose…

Remedies directive.

…remedies directive, pre-sort of financial fallout if that had any impact, but yes, I was just basically seeing “well has it all changed now?” and actually it hasn’t changed, those are the main deterrents to litigation like sort of cost, time, actually there not being appropriate remedies in place or people not feeling that there are appropriate remedies in place or attainable remedies in place, those are all sort of limiting in the same way. Another big factor being sort of the approach the UK has to interim relief and sort of cross-undertaking in damages being a requirement, [?? 16.57] seem to be a real inhibitor.

I find this fascinating because I trained and worked in a completely different jurisdiction, in Portugal is the polar opposite. I mean as a lawyer, doing litigation of public procurement cases was my bread and butter, and it’s probably dozens in a couple of years, and I was obviously only one lawyer, or only one trainee at time, whatever, but that certainly surprised me when I came to the UK and people say, “No, no, we don’t litigate in UK in public procurement”.

Well I suppose you know, the research is basically, you know these remedies, rules, we’re looking at the effectiveness of those remedies why people won’t be suing, but I don’t think that you know, think “Oh, where [?? 17.41]” obviously it’s just me I don’t think but we’re both of the opinion that actually the best system in place wouldn’t be that you know, you’d have something like maybe you have in Portugal or even, I was in Denmark recently and they were talking about how legislatives there, and you know it just doesn’t seem like the right system, but you have just suppliers suing for every sort of minor breach but that’s what we seem to be trying to achieve you know, here.

Yes but the problem with the system like the one we have in UK where there’s limited enforcement in the courts, it means that the contracting authorities can do whatever they want as there’s a limited risk that they’re going to be caught, and that changes the way that people behave.

Well yes and no, I think with these technical rules it’s not always the case though that you know, the example, I was talking to someone in Denmark recently and they were saying how you know just these minor sort of infringements mean that a procurement is just getting shut down for sort of a matter of months as it progresses through the course. You think “Well that can’t be the right approach,” you know where somebody’s not doing something to get around the rules and maybe it’s just because you know procurement, like we said with empirical research you know, it’s not always so exact and you can say that this is, you know it’s not going to flow easily is it, it’s just sort of, it can be quite I don’t want to say messy but you know they’re difficult aren’t they.


And then you know, especially when they stretch over time, so you know whether it should be that a supplier is going to jump on you every step wrong you make just can’t be the right system I think, but it’s the system we have.

And I have to say that I’m a huge fan of the Canadian system of having a procurement ombudsman. I heard Frank Brunetto was the outgoing Canadian procurement ombudsman on a podcast at an early stage and I think it’s a great system actually, it helps defuse a number of potential issues before they reach the court.


In the UK you’ve got the Mystery Shopper service which may evolve in that direction, although I doubt it that they will ever be called procurement ombudsman type of system, but it’s a shame, it’s a shame I think it’s a very good system and in Europe you should look more closely into it. Speaking about your projects what else are you doing?

Well in addition to that we are just starting some further empirical research into the protection of workers on government contracts so I’d like to interview procurement officers in relation to that, and lawyers if they happen to be listed in the UK. Yes just looking at, I suppose just compliance with legal requirements on public contracts, the way in which that might be achieved through the procurement process and on the procurement contract.

Are you’re doing it by yourself or are you collaborating with other colleagues?

That’s just me at the moment yes.

Okay, why do you think public procurement is a good area to mixed-method research?

I suppose because there’s so much to do, you’ve got uncertain rules that you know leave a lot of scope for interpretation, so how are those rules being interpreted? I think just, it’s an important area so it matters to people, so if you’re thinking about public services and having value for money public services, well, and how that effects people whether it be procurement officers, suppliers or the general public, I think those views are important. And it’s an area where it’s just not really been touched by empirical legal researchers at least, so those who actually know something about this complicated area of law and can go and look at it in practice, I know there’s a few business academics who’ve had a look at it, and economists and that’s very interesting, but actually from legal academics it’s you know, sort of, there’s a lot to do because it’s not been covered.

Well, one final question. How do you break down the barriers between different disciplines?

Yes well, as I mentioned before we sort of started recording, I’d be very interested to hear your views on this but I suppose for me I think collaboration between individuals is a good starting point, and I think that’s very difficult but you know, just putting yourself with say a social scientist and economist, and you can both bring something to the table there, but it’s difficult. And you can see actually with some of these research projects that people are forming teams and I think you know, everybody bringing something in from their particular discipline, that definitely seems like the way forward.

Yes I think that’s the way to go. There’s only so much we can do to retrain ourselves to do research that we haven’t been trained before to do, especially as you go and on and onwards with your career and you have certain, let’s say [?? 22.43] costs on a certain career path, you have also other commitments that you don’t have for example while you were obviously an undergrad or a PG student, so it makes really difficult for you to pick up new research skills as you go onwards with your career. So I think in a way I mean, collaboration is certainly a way forward, but the problem I have with collaboration or the problem I have seen with collaboration is that it’s very difficult to break down the silos between different disciplines, because social scientists look at us when you try to do legal research using social scientists’ methods as being second-class citizens because we don’t have as much experience and knowledge about the methods as they have, and it’s a fair comment to make to many of us and it’s [?? 23.27] with economics. So if you try to do stuff with economists, what we can have and I’ve heard this from colleagues of mine which is “What is in there for us?” Because if we want to public in top-ranking economics journals, by and large all of them are pure economics journals that do not touch mixed-methods research so the opportunity cost of doing mixed-methods research becomes very very costly, and it’s very hard to find the right people with the right skills and the right attitudes to break down that silos, because if you think about how we are assessed in the UK in terms of research, we are still assessed by and large in a very traditional silo way. I mean if you’re going to be put forwards for the Research Excellence Framework we’re going to go in the lower panel, and by and large again in general, the people there will have a traditional legal background although actually the law panel is more mixed than others. But on the economics side is just economists, economists and no one else.

Yes I agree, so it needs some sort of fundamental change somewhere definitely, but I think things are changing slowly you know, when you look at what funders and you know, we’re guided by that now impact and funders [?? 24.41] universities, it is just pushing you towards I think collaboration and you know mixed methods yes, working with the different disciplines. So it’s only a matter of time I think.

Okay, one final question, what kind of research or type of research would you like to do in the near future that you haven’t been able to do yet?

Well I suppose without giving too much away, I’m sort of linking back to maybe one of your other podcasts actually, there was, I think you’ve done a podcast with Amy Ludlow.


Yes, so I was really intrigued by that sort of, the ethnography angle, I don’t know what she meant to say sort of [?? 25.20], just observation you know, just putting yourself in the environment so you’re just going beyond say interviews with individuals and for example you know, seeing a procurement process first-hand and, so I’d quite like to do research there, maybe looking at sort of the way in which central governments have tried to achieve some broader policies like localism or big society through procurement and try and bring in some ethnography there, observation.

I think that’s a very good way to finish the podcast, thank you very much Richard.

Oh thank you very much.

You can find me at my blog or on Twitter where I use two handles, @Detig for general discussion and @publicprocure for public procurement related topics. As ever I am grateful for the support of the British Academy Rising Star Engagement Awards which made possible this project.

#13 - Ama Eyo (Bangor University)

Electronic procurement and dynamic purchasing systems

Interview with Dr. Ama Eyo, Lecturer in Law at Bangor University and Director of the LL.M in Public Procurement Law and Strategy.  Ama has done extensive research on electronic auctions in public procurement and has now turned her attention to dynamic purchasing systems.



Let’s start with a small introduction to talk about e-procurement. How do you think e-procurement is going to change in the near future?

E-procurements over the last couple of years, we’ve seen significantly how the attention has turned towards modernising the way procurement processes and procedures are undertaken, as well as in terms of the whole lifecycle of the procurement process end to end, ensuring that e-technology, the advantages that come from it, are deployed within the public sector. Almost in the similar way as the advanced processes we have in the private sector. So looking forward to the next couple of years, what I envision that is likely to happen is more public sector world is going on board with this new appetite, I mean when we look at regimes, such as the former Eastern Europe, they are picking up pace and there are a lot of electronic procurement activities going on there. Then from April next year, with a mandatory e-procurement that’s going to come into force, hopefully, we would likely see more traditional procurement practices giving way to less paper-based transactions and more digitalisation, so that’s something we should be looking forward to.    

It’s not only in Eastern Europe, I mean, Portugal has been using e-procurement as the default way of organising procurement procedures since 2009 I think?

Yes, I mean, the advantage of regimes such as Portugal is that they’re given the kind of learning experiences that the younger generation of EU member countries can look forward to. Practical lessons is to some of the success stories as well as share in some of their faulty experiences in order that those experiences are not repeated in the new regimes, because one of the things that we need to be very conscious about is that the deployment of e-procurement does not, on its own, solve the bad procurement practices, it is an enabler, so we still need to make sure that there’s appropriate training and there’s appropriate specialisations in relevant areas.

How do you think that dynamic purchasing systems are going to fit in in a world where procurement needs to be done online by e-procurement tools?

Dynamic purchasing systems, as we all know, is a procedure which is available for contracts, for works, services and goods, and mainly commonly used with requirements available on the market. Its main distinguishing factor is the fact that as a procurement tool, it is to be used on a completely electronic platform. So for organisations who would be using this tool, it’s possibly the first step towards an end-to-end electronic process engagement that they would be involved in. It could be used potentially as a means of them gaining practice and experience with electronic procurement technologies.

But why would people start with the dynamic purchasing systems if they need to do electronic procurement, why not just adapt existing procedures and do them online?

I think one of the advantages of the dynamic purchasing system is the fact that throughout its lifetime, it’s going to be open and is going to provide access to a pool of contractors. So potentially, what this means is that if you are engaging in a procurement where you want to keep afresh, or you want to keep your options open regarding potential suppliers that may likely become appropriate for that market, this is a tool that could potentially enable you take advantage of that option, because unlike some other practices, I mean, it’s not going to be the solution for all contracts, we all know that, it’s not even being used or touted as such, but it is for commonly required purchases where potentially the contracting authority wants to dispense with a need of going through the whole full tendering processes at each specific time, but then have a platform of potential suppliers, so it gives it that flexibility to bring in new supply base, while at the same time encouraging other good procurement practices if deployed effectively.

True. If my memory serves me well, DPS, dynamic purchasing systems, were introduced in 2004, in the directive 2004-18, am I correct or wrong?

Yes, they were.

So, with over a decade of potential experience in the use of this procedure, why are dynamic purchasing systems not being used more often?

Potentially I think one of the main drawbacks of the old regime on the 2004 directive was some of the requirements in the way dynamic purchasing systems were organised. For example, one of the requirements was that indicated tenders had to be submitted on every opportunity, and also the fact that a lot of systems were not set up at that moment, infrastructure was not ready. At the moment, we do have on the shelf tools that contractor authorities can buy into, that its experience has been gained over the years generally with e-procurements and then the new framework has also, in some ways, enabled the system in a better manner, in such a way that what needs to be done at the outset, it’s to clarify the qualification information of the suppliers and that are admitted onto the framework, with less requirements upfront at that stage.

One of the things I’ve heard on the grapevine about dynamic purchasing systems and why they’re not used as often as they could be, particularly as an alternative to framework agreements is that they imply an increase in transaction costs over the lifetime of the system

Yes. In reality, you will have that drawback still coming in, but again, the decision as to what procedure a contracting authority would elect, it’s based on a number of reasons. Some may be in-house, strategic reasons, and those kind of decisions may be what influence the uptake or maybe they decline. If we look back to things like electronic auctions, for example, we know that when they first started out, because of the government funding, there was support, that was given to support the uptake, a lot of contracting authorities used them, but it’s almost like it’s now on a decline, but again this is something that each contracting authority has to make on the case-by-case basis as to whether this is suitable for them. There are resource implications that have to be looked into. Would a contracting authority prefer to have a standard contract established for a period of years, and dispense with the need to go out to the market every so often, or is it the market’s such there might be benefits in opening up the supply base and inviting new suppliers who could help refresh what is available and out there. So it’s a consideration that needs to be made on a case-by-case basis. Obviously a lot of contracting authorities are treading with care to make sure that they only use it and not create additional burdens on themselves.

True. But one of the things I’ve heard, or one of the things I personally consider to be underlying the reason why framework agreements are more popular than dynamic purchasing systems is that on a framework agreement, you have a higher degree of control, it’s almost the system of set it and forget, you do a selection stage or framework award stage at the beginning, and then the framework is set and is not going to change for its duration, whereas with the dynamic purchasing system, it implies a need and an interest of the contracting authority to be up-to-date with what’s out there in the market and to accept new suppliers into the system.

And that totally aligns with what I’ve just been saying, that it’s a strategic decision that would need to be made by the contracting authority as to what is the imperative for going for dynamic purchasing system. We recognise the fact that it’s one of the advantages of the current regime, is the fact there’s a number of flexible procedures out there for each contracting authorities to choose from, the directive does not mandate as to what must be used on each occasion but each contracting authority takes into consideration what it wants to procure and it’s enabling environment in terms of any national framework that may regulate or implement the directive in future, and that would be the rational for whether it would opt for this or not. Obviously, what we are likely to see in the future is that, for contracting authorities who have not had the experience of monitoring this or using it, the other additional burdens of, and the loss of control which you have mentioned could potentially deter them, but again, what we are seeing in some sectors, especially from local councils here in the United Kingdom, for passenger transport services, is that they seem to find this suitable for that kind of requirement, whether it is the financial agenda or imperative, or whether it is the fact that they want to get, to refresh their market that’s making them go for this, I do not have the evidence to say, but we can see that it’s picking up pace, I mean I’ve been monitoring the use recently, and it’s picking up pace at a much faster rate than before, but only time will tell whether it’s something that is sustainable or whether it’s something that’s likely to die out once the initial inertia is off.

Do you have any numbers?

I couldn’t quantify them but I mean, if you look at current notices, there are about 24 out there.

Which is still a very small number nonetheless.

Yes it is a small number and they seem to be limited to certain sectors. Again, it might be just that somebody’s waiting for someone to use it and then learn from that experience, who knows?

What sectors are those?

The councils are using it, like I mentioned, for, recently home to school or college transportation. With those few examples of set-up or selection by the HMRC, there’s also business and management consultancy related services being set up using this approach, the dynamic purchasing services. Again, you can see that the imperative for this is almost that they don’t want to lock out potential suppliers in the market by using a framework, but then they’re also keeping in case that requirement is not actually used by the end of the day, so it lends itself to that flexibility, and that could be one of the reasons why certain contracting authorities would go for it and others may not go for it, but only time will tell whether the use would gain more acceptance.

Are we seeing a focus on certain contracting authorities or certain sectors?

My research has shown that a number of councils tend to use it, but more towards the Social Services kind of requirements, yes. There’s a couple of community-based interventions that are being procured by this procedure, well, by these routes to market, let me put it that way. Again, you would say that it would potentially be maybe sectors that were under the excluded services regime that are thinking that, ‘OK, we could use this as a way to see what is out there, we know and how to operate things more easily.’

In terms of the advantages that dynamic purchasing systems bring over other procurement procedures and techniques, which ones would you highlight?

I think for me, the most important one would be the wider choice of potential suppliers that are out there, giving the duration of the system, because unlike possibly the framework that you were set up for four years and the suppliers are locked out, this gives you that ability to refresh the pull of suppliers who can participate in that opportunity. Then also, potentially, if in future you have new requirements that can be put in within the current advertisement without breaking the rules on modification or new contracts, that could also be catered for in a more flexible way, so those are some of the key benefits of the DPS.

And on the other side, what drawbacks will you have?

I think, like I mentioned, like you mentioned, the loss of control is there, and obviously if it’s not money taught properly, it could end up in disastrous consequences. I would chip in here with a practical scenario of what happened in one of the cases where this system was used to set up passenger home to school transport system by council. Unfortunately, the nature of the system was such that it meant that vulnerable members of the society were kind of affected by the constant change in the service provider. Essentially, what happened was that the council forgot to exclude the transportation of students with autistic needs from their requirements, and some of the parents complained that it was very difficult for their kids when a different taxi company or transport service turned up to pick up their kids to school. So these is some of the things that need to be looked at, what are the requirement, what are the needs of the stakeholders for the requirements, and how can that be carved out in such a way that while trying to get the imperative of value for money, which seems to be what is driving it amongst some councils, we don’t lose the stakeholders’ needs and concerns in the process.

But that is true for any procurement procedure, I mean, what you’ve just described could have happened as well with the framework agreement.

Yes it could, but you have a longer system of control, but the way this was was that this particular council had a situation where for, it was unable, it actually intended establishing a contract with some transport service providers, but it was unable to do that. The loss of continuity had more effect than if there was a longer term contract in place.

How do you think we could increase the use of dynamic purchasing systems in the near future?

I think one of the good things that we could do, or one of, it’s probably sharing of experiences. In my conversations with some of the councils, one of the drawbacks is that traditionally, people kept doing things the way they’ve always done them, without thinking about how innovative, I mean, nowadays we talk about innovative procurement. Yes, innovative procurement can be, you know, going the long-haul, but maybe, if a rethinking the way we were accessing the market, maybe rethinking is there a possibility for DPS, would it be suitable, that can be one of the ways to improve its, increase its use. Also sharing the lessons, the poor lessons that have been realised from practice so that other councils don’t make those similar mistakes. Personally, my first encounter with DPS was a while back, it was in 2010, and this was before it became really popular, I mean it’s not that popular in the UK but as popular as it is now, I was actually shocked initially when the option was presented on the table as one of the possible routes to market, but reflecting on it and reflecting on the lessons that we learn from that exercise, I can see some of its benefits, this was, we used it to get training associates for teacher qualification training, and it was used to actually revitalise some of the pool of trainers that we had on the contracts with the organisation that I walked with at the time, so I can see some of the benefits, but again, we need to think about how do we deploy it in a way that brings effective realisation for consumers of these services, as well as the contracting authorities.

Just to finish it off, in dynamic purchasing systems, are you including it in the materials of the LLM procurement law and strategy?

Yes, it is part of the module that our students are to, our students take, and one of the things that we’ve done is actually for them to learn how one of the local councils here in Wales, Cardiff Council, has used it practically for Social Care Services as well as the home to school transport, as well as the practical lessons and drawbacks that they can pick from these experiences.

To conclude the interview, I would like to move the discussion into the LLM. So you started, or we started the LLM back in 2011, it’s now 2015 or 2016, in the last five years, what has changed in the field of predicament and in the way that we teach procurement subjects to students?

OK, great, thanks for that question. Uniquely, one of our imperatives in introducing the LLM was to provide a pool of training to experienced and inexperienced procurement staff. Over the years, we noticed that public procurement was taught at the Masters level, the postgraduate level, so one of the things that you’ll be pleased to note is that in order to enable some of the younger participants in our community to graduate from university with some knowledge of procurement, so we’ve got a module, we’ve actually got two modules at the undergraduate module. This is a significant change in that they are being equipped with legal education, albeit from the procurement and the strategic aspects of engaging in public sector contracts, and some of them have gone out to tender which has served them well either in private or the public sector. In terms of the distribution of offerings that we have on the module, the unique aspect of our programme is that we see that, well you cannot do procurement without the knowledge of legal principles. However, legal principles can only take you so far, you also need to have an awareness of some of the strategic imperatives that drive the organisation when engaging in procurement activities, and this is why we feel that there is a need to align both aspects. So in our offerings we combine those kind of issues, so the students look at not only the law but also the corporate strategies, as well as national and international imperatives.

Thank you very much Ama, I think that’s a great way to finish the podcast.

Thank you for having me and do have a good day.

You can find me at my blog or on Twitter where I use two handles, @Detig for general discussion and @publicprocure for public procurement related topics. As ever I am very grateful for the support of the British Academy Rising Star Engagement Awards which made possible this project, which includes both the podcast and also the conference.



#12 - Nat Green (Shaw LLM Fellow at George Washington University)

Ecolabels and environmental considerations in public procurement


The interviewee today is Nat Green, a Shaw LLM Fellow at George Washington University in Washington DC. His publications to date focus on the legal and policy implications of large-scale hydropower development, and other renewables in Africa and Asia, in addition to a more recent foray into questions of sustainability in international procurement.

So, this is the first time that we’re interviewing here someone that is coming from outside the EU, and has a completely different perspective on a public procurement legal framework, so it’s great to have you on board. If I can start to get the ball rolling, what do you think are the main differences between the US and the EU at this level of sustainable public procurement? 

In terms of main differences, it’s sort of an interesting question for me, because I’ve really, as you noted, I’ve come from more of an environment and energy law background that’s sort of, it’s really the potential in procurement mechanisms and you know, government contracting mechanisms in general, to help develop certain environmental, or even social safeguards that’s sort of, drawn me to this subject. So it’s actually the similarities that I’ve concentrated on more than anything. In terms of the differences, I do think a lot of it’s political. I think there’s more of a political will in the EU to open up the process to what we call our economic externalities, where we might also call, in much of the literature, market failures. In the US that is, we share many of the same principles, and providing for open competition and, sort of, fair value at pricing, and in-state procurement, but as I said, the political will to provide for a more open interpretation, and incorporation of externalities into the system is, it’s a little bit more difficult to attain I think, which is one of the reasons why I’ve come to focus on the potential in eco-labelling systems. You know, going forward, especially in this, and providing kind of, a shared sustainability network for the two models.

So, what are the similarities then between the US and the EU, in what concerns the use of eco-labels? 

Well, interestingly, there’s certain, what I have come to think of as flexibility mechanisms or models that are built into those systems. The thing with eco-labels, you know, this is essentially something that allows us to harness market forces. One thing that the US and the EU share of course, is a push towards ever greater market liberalisation, and the idea is of course that if you have a third party, usually non-profit, that has a mandate to certify certain practices or services or products, as, you know, having sort of, a lower impact on the environment, it allows consumers to use their purchasing power essentially to help to achieve the goals that lie behind the certification. Right, so the similarities are pretty much found in what I’ve, and many others have called the Or Equal Standard. Well actually, that’s how you use it.

The similarity in the legal approach to eco-labels is actually, we have very similar statutory language. For the US FAR, the Federal Acquisition Regulations in, let’s see, that would be FAR 23.103, we actually, it encapsulates a series of recent executive orders that call for the greater use of eco-labels in government contracting. Actually it calls for very wide spread use, and you know, as I was trying to say earlier, the eco-labels, it’s an attractive way to do it, because like I said, politically it’s very difficult to even establish what sustainability is, and in fact the word sustainability, people shy away from it a little bit now in the environmental law world in the US. You know, here in Washington DC, the Environmental Law Institute, if you go and talk to the scholars there, they’ll tell you how about ten years ago it became a really big idea, and people just got tired of arguing over what the word sustainability even meant. These eco-labels kind of fixed that for you, they make the decision for you.

So, that sort of, jumps us past the political and policy wrangling over many steps and deciding what we mean by sustainability standards. So, the US FAR now provides that government contractors need to use products that are certified under certain labelling schemes, and similarly recent EU directives have called for a great incorporation of eco-labels, and some of these issues have also been incorporated in the GPA, so what really brings them together is the way that they allow the incorporation of eco-labels in such a way that anti-competition regulations are not upset.

There are a few points there for us to unpack a little bit more. I’m fully behind the idea that the word sustainability became pretty much meaningless, because it’s more of an umbrella term, or another term where you can project whatever you want into it, depending on your background, depending on your policy objectives, so on and so forth, and that’s not something that I originally envisioned, you know, or anticipated for that particular word, so I completely agree with your view that it’s preferable to use more precise terminology like eco-labels. Having said that, one of the fears that I have, in what concerns the use of environmental considerations in public procurement, and also social considerations in public procurement, at least here in Europe, is that they can be used in a way that it’s not market friendly. They can be used to actually restrict competition and restrict the market to certain suppliers. 

Right, that’s a concern that I’m aware of, and I actually, I find it really interesting, because from my perspective, the use of eco-labels, it’s an interesting tension, because you know, on the one hand we’re trying to, if we want to start using eco-labels as a way to further policy ideas, and specifically sustainable policy ideas, it has to work as a market instrument. As you say, you know, many have raised the possibility that it could actually become, you know, certain member states in the EU for example, could start favouring certain eco-labels, and that could actually become a barrier to cross border traffic. I know that some… the idea’s been floated so that in future directions, or in future revisions of the GPA, they include competition guidelines that specifically address the use of eco-labels to counter against this. I can’t be sure how successful that would be. Again, from my perspective, as being a US environmental lawyers, and especially, you know, I have a strong interest in climate change issues, I don’t see the eco-label as a permanent fix.

I think that many scholars on the European side have looked at the potential for, sort of, long term systemic change, essentially to procurement culture, in providing more, kind of, sustainability safeguards, but I think they’re very correct in addressing the fact that this would require tiny, incremental steps at every stage of implementation, because you’re changing the driving forces in many ways. You know, you’re having to redefine just basic issues of value. The use of eco-labels, it’s sort of, a low hanging fruit. You could even see it as triage. It’s a way to help to start shifting certain, really commercial standards while other policy are implemented at sort of, a higher level. I guess I see that, sort of, two-stage process as having some potential to alleviate any anti-competitive side effects, if that makes any sense. It’s just a, it’s an idea I’ve come up with.

Well it does to a certain extent. I mean, on the other side of the argument you have the idea that eco-labels themselves may be considered anti-competitive, because effectively you’re giving a monopoly to an external organisation, even a standards setting body, whatever, to define what are the requirements that everyone operating in a certain market will have to comply with, because you’re requiring that eco-label, whatever it is. And if you think about the Max Havelaar case, let’s call it the Dutch coffee case, so it’s easier for the people that are going to listen to this, so if you think about the Dutch coffee case, the crux of the matter was precisely that it was a private standard that was being imposed on the market. And that was deemed to be anti-competitive, and that was seen to be as not compliant with EU law. 

Right, right, and one thing that the court did with that decision, is they made a very small observation, which is that if, you know, if the contractor had simply included ‘or equal’ language in the contract… they included it in an annex to the contract, but that wasn’t clear enough, and I would agree with that, meaning that this private label became by its inclusion in the contracts, became not so much the label itself, but represented a certain standard. Now in terms of, you know, going forward, coming up with a standardised way for contractors to take a given eco-label, and decide exactly what standard it represents, that’s another issue, and I haven’t hit that yet. But that’s the decision the court came to, subsequent your EU directive essentially took that and tried to… said that if a private label is incorporated as a contracting requirement, then it’s to be taken automatically as representing a standard, rather than that label in particular. You know, that’s the stopgap measure in terms of preserving competition. And the US has this too.


You know, what I agree with is that it provides an opportunity for harmonisation between EU and US procurement systems specifically dealing with environmental issues. You know, I think that that’s a potentially powerful tool in providing, kind of, a shift in industry standards in quite a few different industries. I’d be interested to see how that worked out.

Okay, so if you dig a little bit deeper into that, how should the US and the EU change their use of eco-labels in public procurement?

Well, as I said, I think the next step is to see how this Or Equal Standard actually works out. You know, as you say, whether or not bodies in the EU and the US can, in a reasonable and cost effective and efficient fashion, take the principles inherent in private eco-labels and apply them as more general standards, that might require a certain growth in institutional capacity. It also will require close attention to its competitive effect vis-à-vis small and medium enterprises. I know less about this in the EU, but in the US of course we… that’s one area where a certain degree of, forgive me, anti-competitive… you know, preference is allowed, and because it’s seen as an important policy issue.

Can you tell us more about that?

Well, off the top of my head, the actual FAR part, small and medium business considerations, or small and medium enterprise considerations are, essentially it’s almost like we have an affirmative action to allow small and medium enterprises to compete on a more equal footing with your larger government contractors. And of course that shifts with the nature of what you’re… of the product. You know, we’re not going to work very hard to make it so that a mom-and-pop business can try to supply military hardware, although there are very small businesses that require less intensive, you know, military hardware that it’s less difficult to create, or make or research. I know that there are very small body armour companies for example, that are contracted to the US government. But you know, beyond that, when it comes to your basic supplies, say your coffee, or you know, services, people shoot for an equal playing field. So, obviously that policy consideration is still linked towards the whole idea of greater competition, so it’s hard to make a direct comparison, I suppose.


Moving forward to another topic, revisiting something that we’ve already mentioned, but not in detail, which is externalities. What is your view about the inclusion of externalities in public procurement? Pollution is an obvious example, but what else can you think of? Should we go down that route or not? 

I’m not sure. Like I said, this is actually, fortunately your podcast is about, you know, emerging academics. This for me is a very emerging field. It’s actually a little bit difficult for me to generalise. I love the idea of using procurement and using these kinds of tightly controlled markets to further certain sustainability goals, and I see it as being, kind of, the opposite end of the spectrum from say, the Paris COP21 talks that are happening at this moment. I realise this is sort of, tangential, but just to take a quick observation, is that for the COP21 these talks are… it’s an old, established model for reaching international agreements. When it comes to environmental issues, the great success historically of course is the Montreal Protocol that eliminated… that gradually scaled down and eliminated the use of CFCs in aerosol cans, and had a very positive impact on preserving the Earth’s Ozone Layer.

With greenhouse gases it’s really, really a very different story because you’re dealing with a much greater range of practices and technologies, and they’re much more vital to the way we live our lives. The interesting thing to me about trying to [?? 14.51 attack] any externality, or you know, like I say you could call them market failures, through procurement is that it’s immediate. It can have an immediate and very wide ranging effect, on a wide range of industries. I was fascinated when I found out that Walmart here in the US is one of the largest government procuring providers, or suppliers.


Yeah, so I mean, think about it that way. Anything that… government procurement takes up enough of their businesses that it wouldn’t be cost effective to just shift part of their business towards requirements to bring them into compliance with government contracts. They’d probably have to do it across the whole range of their business.

That reminds me of something that happened maybe ten years ago, with the Reduction of Hazardous Substances, whatever Directive, which when it came into force and effectively led to a change in how suppliers based anywhere in the world could sell their products here in Europe, their electronics products, what I heard was that effectively there was an externality coming out of that process which led to the whole world benefiting from the changes imposed by the EU, because it became economical for a supplier to run two, effectively two different, let’s say production lines, one for Europe one for the US, or one for Europe and one for the rest of the world.

No, that’s exactly it, and actually on another angle, one of the reasons that I believe, I just, I’m… I’m pretty sure about this. One of the reasons is that the US has very low standards for cosmetics, especially perfumes.


Yeah, and perfumes can potentially have quite scary chemicals in them. They’re all constructed at this point from petrochemicals, and there are a lot of things that can go inside that are, you know, I mean, from my point of view a little bit unchecked, essentially. And, you know, we really don’t have much of what we call a cautionary principle. We don’t err on the side of caution, we err on the side of, you know, people exploring their options, which you know, has its plusses and its minuses. And the thing with perfumes is that most of them come from Europe.


We are piggybacking on your standards.


There are some… anyway, that’s completely tangential, but I find that very interesting, and yeah, at the same time I think, you know, we do allow for a certain amount of anti-competitive legislation where the public interest is really a concern, and hazardous materials is a perfect example. You know, in this case it’s slightly different, because we’re trying to wire externalities into a legal mechanism.

Yeah, yeah, of course.

The whole purpose of which is to uphold a certain level of competition. So I realise this is tricky, I really do, I just wanted to show how it can be done potentially, without breaking too many laws.


 Yeah, or any!

 Or any, or any. You know, just change the law, and that’s also, we’d [?? 17.59] to do.

 Well, I don’t think we really do it at this point. I think you’ve got to actually employ eco-label systems for products and sell them pretty much effectively the same way between the US and the EU. I think it would lead to some cases certainly.

Yeah, I think so. Speaking of both jurisdictions, or both legal systems, we have something in the horizon coming fast towards us, or not, the TTIP, the Transatlantic… what does the acronym mean? I never know.

Don’t ask me!

Well, let’s just call it the TTIP, everyone knows the acronym, and no one knows what it means, or at least I don’t, I remember out of the top of my head. Anyway…

Yeah, Transatlantic Trade and Investment Partnership.

That’s it. 


So should we consider, or should the negotiators in both sides of the table consider using the TTIP for example, to harmonise eco-labels across both the US and EU?

I think the potential is certainly there that we could. I understand that the TTIP negotiations between the US and the EU are based on the revised GPA. The revised GPA does include provisions for the inclusion of eco-labels, and includes this, you know, again the Or Equal Standard. The Or Equal Standard is simply just that when you write the contract, when you draft the thing, when you say, like, I want, you know, I want my coffee to be Fairtrade certified, you have to say Fairtrade certified, or equal.


Right? So, just to make that explicit. It’s there in the negotiations right now. It’s on the table, so that’s something that I feel could de facto harmonise the use of eco-labels in procurement, because I think that the legal structure internally in both the US and the EU is there to support that.

Okay, so if you think about an example, let’s say that we have an eco-label in Europe, and another one in the US, they’re not exactly identical, although they cover the same area. I mean, the problem with the ‘or equal’ approach, which I mean, it’s very common in Europe in many areas related with procurement, it’s not just with the eco-labels. The problem with that is, where do you draw the line on the equivalent?

I agree, I think that that’s the biggest stumbling block, and I think that honestly I’m not sure yet. I’d love to hear any ideas that anyone out there might have. You know, off the top of my head, that’s something I’ve been keeping as kind of, an open question. I think that in my dream of an ideal future, if eco-labels do become a means of commerce driven dialogue between the US and the EU, I consider it possible that the question, the tension that will inevitably be there regarding exactly as you say, how do we express individual eco-labels, and they are very individual, as given standards, will lead to greater legislative and policy dialogues. You know, I see it as a potential legislation and policy driver. That’s not to say that I think we should… I’m thinking of this as a monkey wrench that we throw into the machinery, you know, not at all. But it’s something that I think has been developing for some time. The EU itself has been putting significant resources into developing a very reliable EU-wide eco-label. I think that policy members, especially in the EU have had this transition, or this possibility of sort of, the basis for a dialogue, in mind for some time, and in the US we’re just beginning.

Eco-labels for us have usually involved on the legal level, have usually involved issues of international trade under the WTO. You know, we have the great tuna-dolphin, and shrimp-turtle cases. Tuna-dolphin at least is still on going, amazingly. And so in terms of using it specifically as a part of government administration, that’s a new idea. It really depends on probably who our next president is, how far we go with it. If it’s a Republican, not so much, if it’s a Democrat, maybe.

Okay, I think that’s a very good way to end the interview. Thank you very much for your time Nat, it was great having you.

I had a very good time, thanks for having me.

Before I close the podcast, I’ve got an announcement to make. You can find information about the Early Career Research Day Conference at the website. Long story short, if you’re interested in presenting a short paper, please submit an abstract and CV by January 11th. We’re aiming to run the conference, I think on March 4th, which is a Friday, and it’s going to be done in London. We will be selecting ten early career researchers to take part in the conference, and the travel costs will be covered up to a maximum of £300. We will also offer one night accommodation at a known sumptuous hotel in London, so if you’re interested, just drop me a line, or submit an abstract and a CV. You should do so using the contact form available on the website. Finally, as usual you can find me at my blog or on Twitter where I use two handles, @Detig for general discussion and @publicprocure for public procurement related topics. Finally, I’m very grateful for the support of the British Academy Rising Star Engagement Awards, which make not only these podcasts possible, but also the conference.

#11 - Abby Semple (procurement consultant, postgraduate student)

What will public procurement look like in 2025?

Interview with Abby Semple, a Procurement Consultant and postgraduate student at the University of London Birkbeck College, who runs the blog Public Procurement Analysis. She’s also an expert in sustainability in procurement. Earlier this year Abby was one of the first authors out of the gate with A Practical Guide to Public Procurement, a book about the new public procurement directives. There were many topics we could have chosen for our talk today but we settled on one slightly different from usual. Today’s talk is focused on the future of public procurement, more specifically how procurement may look in 2025.



Hello Abby, thank you for coming to the show.

Thank you. Nice to be here.

We were discussing when we were setting up the interview what topics we could cover and what questions we should go for. One of the first ones you suggested I think is very, very good, if you look ahead for the next ten years or so what would we perceive is going to be the changes to the market in public procurement in Europe? Will there be more or fewer contracts advertised and more or less competition?

Well it’s maybe a bit of a risky topic for me to have proposed because I think most people throughout history who have tried to predict the future have been proven wrong! It’s often interesting to see in which ways they were proven wrong so it’s maybe a little bit dangerous to talk about this but never mind, you know, it’s a Monday morning so may as well get stuck in. In terms of that question of more or fewer contracts, for me the big question is at what level are contracts being advertised? Are we just talking about OJEU advertisements or are we talking about national databases, national websites? And my guess would be that you’re going to have about the same number of contracts advertised in the OJEU but a lot more contracts being advertised on national websites at national level and a lot more potential at least for cross-border competition via those national portals as they sort of gear up. And I don’t know whether they’ll all be following similar standards but at least they’ll become more intelligible, more accessible to bidders outside of the Member States where they’re being advertised.

So you reckon that it’s not going to be a big change above the thresholds and if there’s to be any change at all it’s going to be below thresholds?

Yes, that would be my best guess. Because if you look at what’s advertised in the OJEU at present you see some really interesting trends in terms of which countries are advertising the most contracts and it tends to be the newer accession states who are advertising a lot of contracts and many of those are below threshold contracts. So I think in some cases there’s a little bit of over-anxiety about advertising. In some cases those advertisements might be required because they’re receiving EU funding for a specific project or contract but if you look at some of the older Member States, like Germany for example, Germany advertises a very low number of contracts, and that’s partly because they have a very decentralised procurement system, but I do think you find that as time goes on countries get used to the idea of they don’t actually need to advertise every contract in the OJEU and as procurement potentially becomes a bit more competitive below threshold within a Member State, you see “okay, we’re actually getting adequate competition by advertising at national level”.

In terms of the national advertising of contracts, do you reckon that just by the fact that those contracts are going to be advertised, albeit in a national portal, those contracts will more likely be subject to cross-border interest and more likely may attract actual interest from cross-border economic operators?

Yes I think the potential is there. And again if we’re going to get our crystal balls out we need to think about what are governments going to be buying in 2025? And we’ve certainly seen a move over time, governments to some extent are still buying hard supplies but there’s been a general move towards buying services over the past fifteen, twenty years. And that partly reflects the fact that certain functions of government have been privatised or they’ve been partially privatised or that things are being outsourced through service contacts, whereas previously it might have been a supply contract with the service element being provided in-house by a public authority. So that’s been a trend in quite a few European countries and I think you need look at the nature of the services being advertised.

We sometimes talk about cross-border procurement as if it’s just a question of access, that companies can know about contracting opportunities and then they’ll bid for them but of course they also have to actually be able to deliver those contracts. So if it’s a type of contract like a social care contract where you very much need to have a strong presence on the ground, you need to be able to work with employees in the location where the contract is going to be delivered then I kind of doubt we’re going to see huge amounts of direct cross-border procurement for those type of contracts. On the other hand we see a move towards things like printing, digital services, data services, all of those obviously have huge potential to be outsourced on a cross-border basis so I think we will see more in that sector. So the question of the overall amount of cross-border procurement that we’re going to see depends first of all on what type of contracts are being advertised and then, secondarily I think, on how accessible are those contracts to bidders in other Member States.

I think you’ve touched on a point that is very important which is the one about the kinds of services that are being procured right now or are going to be procured in the future. I mean if you look at the development of digital services in general we see that their importance has been increasing in terms of GDP, in terms of percentage of GDP as time goes on and it’s not going to stop there, so it’s just a question of time to that kind of influence to start to be seen as well in public procurement. So one of the things I think will happen in the near future is that we’re going to have a lot more digital services being acquired and being procured, and by definition those digital services by and large come in at values well below the current threshold levels, so that’s one of the discussions that I’ve been pushing forward over the last couple of years now which is what should we do to the thresholds going forward?

I’ve read some of your work on that and I think it is quite a big point to raise: what is going to happen with digital services and should we be looking at lower thresholds, should we be getting rid of the idea of thresholds? I think again being realistic about what’s going to happen over the next ten years it’s probably unlikely that the thresholds are going to go way down, partly because they’re linked to international agreements that are in place, whether it’s the WTO Government Procurement Agreement or these bilateral trade agreements which are potentially going to come into effect over the next few years, and I think there would probably be a reluctance to lower the thresholds if that’s going to be then passed on to third countries as well. So whether we start to look at a sort of two-tier threshold system where one threshold applies in the European Union and another in respect of third countries, I don’t know whether that’s realistic. But I think beyond the issue of “okay, are these contracts subject to the EU rules? Are they advertised at EU level?” I think there is a more fundamental issue in terms of digital services and in general ICT contracts, are they appropriate for the way public procurement runs, this idea that you can sort of have a competition and define outputs and award a contract and then sort of stick to those outputs. And I think for some of the more straightforward contracts that’s fine but increasingly we find there’s a long list of failed digital services or failed ICT procurements, not only in the UK and Ireland but elsewhere, so I think that is a real challenge for the rules and how we apply the rules to those type of contracts.

I remember having a conversation a few months ago or last year with Frank Brunetta  the Canadian Procurement Ombudsman and he was making a suggestion that actually makes a lot of sense, which is if you think about it the way that procurement is run today it’s based on premises and ideas which were designed to allow for the procurement of goods and works. And that is a very different kind of exercise that perhaps the procurement of services would require?

I think that’s absolutely right and you still see that. Maybe a little bit less so in the 2014 directives compared to their 2004 predecessors but it’s quite clear that they’re written from that point of view, of being able to define an output, of having a pretty good idea of what it is. That said, we have seen the introduction of the Competitive Dialogue and more recently the Competitive Procedure with Negotiation. Competitive Dialogue in particular, as you well know, is designed in particular to be appropriate for those type of contracts but unfortunately we’ve seen a bit of a backlash against it in the UK. There are a lot of countries where it’s never been used at all or used only very rarely which I think is a real shame because it does have the potential, for ICT contracts or complex services, to be the right procedure.

I agree with you. Moving onto the second topic, what kinds of award criteria and procedures do you think will be the most common?

One of the things that I have as a big question mark in my mind, because it’s an area where I’ve done a bit of work recently, is this idea of life-cycle costing. It has always been possible, if you’re using most economically advantageous tender as your award criteria, to apply a life-cycle costing approach. What we see in the most recent European directives is that there’s been an attempt to set more detailed rules around how you do life-cycle costing, what information you can ask for and there’s this concept of data that can be provided “with reasonable effort by a normally diligent operator”, which I think will be an interesting one if it gets litigated, which it probably will at some point in the next ten years. So that’s one question in my mind, are people actually going to use life-cycle costing or are they going to be scared of it by the fact that there are more detailed rules around it and that there is a potential for an operator to challenge the use of life-cycle costing if they don’t like the outcome. I think what we’re seeing across industries is that supply chains are getting more complex, that the level of data that people are looking for is really unprecedented, so it is a challenge and it’s a challenge which some companies are very well aware of and are working hard to address but obviously not all of them.

Do you think that lifecycle costing is going to be used a lot over the next decade?

I think there will be a desire to use it. I think it’s something that people are aware of. I think it makes economic sense as well as environmental sense. So as procurement becomes more professionalised, as it becomes a bit more sophisticated definitely the idea of awarding a contract based on purchase price alone is going to become a bit of anachronism except for maybe some very basic types of supplies or commodities. So I think in general we’ll see more of it but there’s this question of are people going to call it life-cycle costing? Are they just going to say “look, here’s our form of tender and we want you to cost the following eight things” and not refer to it under this idea of life-cycle costing?

In terms of procedures, which ones do you think are going to be the most common?

The open procedure I think will continue to be used. There are people who say the open procedure is too basic, it doesn’t make sense but I think, the open procedure is always going to work well for certain types of requirement. And we know that at the moment it accounts for about three quarters or at least two thirds of procedures advertised in the OJEU.

Except in the UK?

Except in the UK. So the UK and Ireland have always been a bit of an exception to that, there’s been a preference for the restricted procedure. It’s interesting because some of the figures I’ve seen suggest that one of the reasons for that is that, particularly in Ireland,  is that procurement tends to be more competitive, if you’re running an open procedure even for a relatively low value contract you could be getting thirty or forty tenders and some of those will be cross-border tenders because of the fact that we’re running procedures in English and quite a few Europeans now have English as a very strong second language. So there is an experience of receiving more tenders. I think for that reason local authorities, local authority procurement tends to be a bit more competitive than central government procurement, they have said “right well we’re going to use the restricted procedure because we just don’t have the resources to deal with assessing thirty or forty tenders every time we procure a relatively small value requirement”.

I think that that tendency will continue to exist but the thing that’s changed under the new directives is that for the restricted procedure, you have these more extensive publication requirements at the beginning of a restricted procedure. So if you look at Article 53 of the Public Sector Directive it says that you have to have the procurement documents “fully and freely available online from the date of a contract notice”, and it’s a little bit ambiguous as to whether that includes your invitation to tender which formerly would have been a second stage publication, but now it looks like you have to publish it at the outset unless you have a reason for not doing that. So I think perversely that might actually encourage people to go for the open procedure because they’re going to say “well we’re going to have to publish everything at the outset anyway so we may as well just go open procedure”.

I’ve got a comment about Ireland, I understand what you say in terms of the language and it makes a little bit of sense, however I mean tenders here in UK are also in English and the UK is probably one of the member states with the lowest levels of cross-border procurement, are foreign economic operators actually winning tenders in UK?

I think we do have to take the figures on that with a grain of salt.

I know.

I cite them and you and everyone else cites them but I think in general we’re talking about that one study that was published in 2011 on cross-border procurement and while I think it’s very valuable to have that study, even within that we saw there are issues with methodology in terms of sampling, there are issues in terms of the quality of information we’re able to get from OJEU award notices. So I think it’s probably accurate to say there are not huge amounts of direct cross-border procurement happening. When you get into the more complex questions like “What about indirect cross-border procurement? What about use of subcontracting?” I think we definitely do have to take those findings with a grain of salt. That said, it probably is true that in the larger Member States like the United Kingdom you’re always going to have lower levels of cross-border procurement because you simply have a bigger domestic economy and you have a greater chance that economic operators will see it as being worth their while, if they’re serious about tendering for government business, to set up an office in that Member State.

And also it’s more likely that you’re going to have a supplier inside a country that’s going to be competitive enough to win the contract?

Exactly. So it is interesting again, while accepting that we can’t take them as gospel, to look at the findings from that study that you do see patterns. Ireland has pretty high rates of cross-border procurement, partly because there are two countries on the one island, so obviously there are Northern Irish companies bidding for contracts in the Republic and vice versa, that automatically puts the numbers up. But then you see countries that share a language, in Germany and Austria you see slightly higher levels of cross-border procurement between them. So there are all kinds of interesting patterns that give you an insight into where this is happening and perhaps why it’s happening.

Okay. Let’s go on to the third topic, will procurement challenges be more or less frequent?

I think this is one that obviously is of interest to the lawyers but also of interest to contracting authorities because there is at the moment a big discrepancy in the frequency of challenges between Member States. It’s an issue I looked at a little bit in my book, I focused particularly on the UK and Ireland and I think the major thing we have to take into account is the cost of bringing those challenges. And for as long as you have a system which requires bidders to bring a challenge in one of the higher courts that’s going to be extremely expensive, and even though the threat of procurement challenges might always be there, the actual number of challenges which make it through to Court is going to be reasonably low in those jurisdictions. In a way that’s kind of beside the point because the thing about procurement challenges is that a lot of it is hidden, we don’t see the letters that contracting authorities receive, we don’t see how they react to those letters for anything that falls short of court proceedings usually in the United Kingdom and Ireland. Then in other Member States such as for example Sweden where they have a relatively accessible means of challenging contracts, you obviously see much higher numbers, but at the same time I don’t know whether the threat of challenge is really taken as seriously by contacting authorities. That’s maybe something you could talk to Andrea Sundstrand or someone else about, although I think you’ve already interviewed her haven’t you?


Because my feeling is when I’m working with a client in the UK or Ireland and they are potentially on the receiving end of a procurement challenge, that’s something they take extremely seriously and often they’ll decide to cancel a procedure and start again simply to avoid having to go through that lengthy process of challenge. And I don’t know whether that really applies to contracting authorities in countries where the remedy system is not as expensive, perhaps not as big of a deal essentially to undergo a procurement challenge.

And to my mind that’s actually a bit of a healthier system to have, to have a system whereby it’s relatively easy for economic operators to bring a challenge but it doesn’t have the huge cost and time implications that a procurement challenge does in the UK or Ireland because, let’s face it people do get things wrong, the remedy system is there to ensure that there is an avenue of redress when things do go wrong so you just want to make sure it’s not abused and that it’s not used as this sort of nuclear threat which I think it is in the UK and Ireland.

Yeah, I think that’s a very good point because I’ve got experience in other jurisdictions namely in Portugal and Spain and the normal thing is for every single tender procedure to actually be challenged.

Right, okay.

So you just take it for granted and if you don’t get a challenge, well that was a good day for you. Whereas here in UK the perception, it’s more a cultural issue as well, which is if you get a challenge that is perceived as being a black mark, you made a mistake as a procurement officer that’s why you got the challenge, whereas in other countries it’s just the normal way of doing things. As for Sweden and Denmark to a certain extent they have remedy systems which allow other avenues for bidders to actually try to interfere with the process in a sense that if they think that something is going wrong or went wrong, so I think it is actually the Swedish Competition Authority who has the power to actually intervene during the procurement procedure. So that changes the dynamic a lot and the fact that you can have different kinds of systems remedy procedures which are different from just going to the course actually probably allows those systems and those procurement frameworks to work better. Another good example is Spain, Spain a few years ago they introduced a new review system or review mechanism whereby you could have access to administrative tribunals, literally independent tribunals are not dependent as they were in the past many, many years ago, the fact that you can have a quick decision taken in a few months with a price that is reasonable actually has improved the way that the procurement market works overall.

That’s interesting. A quick decision and also potentially one by someone who understands procurement…


…because they’re dealing with it every day and, with all due respect to judges in the UK and Ireland most of them are not dealing with procurement challenges with any type of regularity. I mean we see now with the Technology and Construction Court in the UK that there are a couple of judges who have developed that expertise but it is a difficult area I think. And judges are quite upfront about that sometimes, they say “I’ve had to go away and read hundreds of pages about public procurement and I’m still not sure I’m applying the right approach here” and that I think is not an outcome that’s in anyone’s interest. It’s a huge use of resources in order to resolve what sometimes look like pretty stupid claims to be honest, or very minor points about “did this person score this correctly?”. And there’s no implication sometimes that anyone has acted corruptly or that they’ve even really committed a serious breach of the rules, just that there was some kind of basic error that happened but it takes so long and it takes so many resources to resolve that error, and then what is the outcome of that challenge? It doesn’t necessarily mean that the challenger gets the contract, they might get damages if they’re lucky and the authority might have to re-run the procedure. So I think the ratio of costs and resources going into procurement challenges versus what they’re doing to improve outcomes or to remedy problems that have occurred in procedures is the balance, we’ve got it wrong at the moment in the UK and Ireland and I think we would be well advised to look at systems that are in place in other countries. Even potentially the Procurement Ombudsman system that they have in Canada and other countries.

I’m a huge fan of the Procurement Ombudsman system and I think that is one of the best. Unfortunately I don’t see the UK adopting it anytime soon but that’s my take. One final question, Procurement of Innovation, is it the idea of the future and will it always be an idea of the future?

It’s got a bit of both attached to it. I think for good reasons because particularly coming out of the financial crisis in the European Union, there was a need to do more with less, government went through a bit of an existential crisis in a lot of countries, what is our role? Innovation that really works is something that everybody wants and everyone can agree on. What it actually means in practice I think is a bit more difficult and I have been through a couple of innovation procurements recently where it’s extremely different, it’s 180 degrees away from normal procurement where you know what the outcome is. People talk about using functional or performance-based specifications, I mean that’s fine but you still need to be able to evaluate what bidders are proposing to you, you need to be able to structure your contract in a way that creates the right incentives on both sides. So yes, I think there is a capacity being built up to do it but I don’t think you could say anyone is really in the business in Europe of innovation procurement on a regular basis or at least I’m not aware of it. And people often cite examples from the US about the work that NASA or the Department of Defence has done that led to the development of the internet or that the health research networks have done, and it’s interesting on an anecdotal basis but I’m not sure it really translates as a model that can be adopted on a mass scale. So it will be interesting to see what comes out of all the sort of Horizon 2020 funding because there’s a lot of that going round at the moment but I guess I’m a little bit of a sceptic about the ability of the EU funding programmes to create a culture, cultural change. I think they can certainly be influential at the level where people are able to do things they wouldn’t otherwise have been able to do. But if you’re trying to push people into what is quite a profound cultural change, which I think innovation procurement is compared to normal everyday procurement, that takes a lot of time and ongoing incentives rather than just having a one-time access to a European funding stream. There needs to be support at local level, at national level, and there needs to be an understanding of, “What is this? Is it valuable? Is it something that’s going to get us towards our long-term objectives?”

I think we still have time for one quick final question. What would you like to see changing between today and 2025?

Well a lot! But since we only have…

Just one idea?

One idea? I’d like people to be less afraid of the procurement rules. I think there has been an over-legalisation of procurement rules. I think it’s become way too complex. I’d like people to be comfortable that they can procure something, they can get the right results without breaking any laws and if that puts a few of us who are procurement lawyers out of work then so be it. I think it’s more important that when public money is being spent people have the confidence that they can do the right thing and that they’re not going to wind up in court or hurt.

Brilliant. I think that’s a great way to finish the programme, thank you.

Thank you Pedro.

You can find me at my blog or on Twitter where I use two handles, @Detig for general discussion and @publicprocure for public procurement related topics. As ever I am grateful for the support of the British Academy Rising Star Engagement Awards. 

#9 - Franco Peirone (University of Piemonte Orientale)

How can we minimise corruption in public procurement? A look into the USA, UK and Italy



Interview with Dr Franco Peirone, a Postdoctoral researcher at the Department of Business and Enterprise at the University of Piemonte Orientale in Novara, Italy. Franco’s carried out his doctoral research on corruption in public procurement and remains an active researcher in this area. It is no surprise then that the topic of the conversation is corruption in public procurement, in particular the experience in the USA, UK and Italy.



Thank you very much for accepting, and also thank you for accepting to speak at such a late hour. It’s almost like, as you said, a late night David Letterman Show.

It’s true, but it’s good.

Let’s start with a short introduction about yourself and your PhD research.

I started my PhD research at the University of Turin. Everything hasstarted during the economic crisis in 2011, I was just come back from the Maastricht University where I spent a period within the Erasmus Program, and the focus on Italy was really high, because everybody was scared about Italian historical deficiencies, such as corruption or inefficiencies in the public sector, so I tried to discover what really went wrong in the Italian public sector, and I decided to draw my attention on public contracts and corruption. To this end, I decided to focus on anti-corruption systems and particularly on an anti-corruption system that really works well, as the United States was, and for this reason, I spent a visiting period at the George Washington University under the supervision of great teachers, as Professor Schooner and Professor Yukins, and there I developed some conclusion about corruption, public procurement and public policy.

Once I came back to Italy in 2014, I was able to compare these two anti-corruption systems and focus more on corruption in public procurement. What does it mean? Which kind of frame we can think about it, and which is going to be the next direction of public procurement and anti-corruption tools? My PhD thesis, at the very end, really focused on compliance and ethics programs. They could be considered a broad and flexible anti-corruption tool. It is really interesting that anti-corruption programs, as compliance and ethics programs, moved from one legal system to another one, from the United States they have been spread all around the world and they have been implemented in Italy, in the United Kingdom, in Germany as well as in Japan and in Australia. It’s really interesting also noting that anti-corruption programs have been moved twice; there has been a double legal transposition, for instance, in Italy. From the United States to Italy, and from the private sector also to the public sector. So now we’ve got compliance programs both in public and in private sector in Italy. 

Let’s focus for a minute on those compliance programs. What do they entail? What’s included in them?

Compliance programmes are an interesting topic, because they bridge together criminal law, administrative law and public procurement law. Compliance programs were born in the United States legal framework for avoiding the corporate liability in case of an employee misconduct. The legal assumption at the basis of compliance programs is the criminal law principle of corporate liability. Due to that, if an employee commits a crime, he acts on behalf of the corporation, and if the crime can advantage the corporation itself, also the corporation has to face a trial. Corporations are therefore held accountable for the crimes committed by the employees unless an effective and compliance programs is implemented. To this end, a compliance programs consists in a set of tools such as code of conduct, training programs, auditing, reporting, disciplinary measures, directed to prevent and to repair the employee misconduct such as corruption, for example. By criminal point of view, a criminal law point of view, by adopting an effective and compliance programs, corporations may demonstrate to have used all the force needed to prevent misconducts or crimes such as corruption.

Do you apply the compliance programs before the crime occurs, or after the crime occurs as a means to reduce the impact of the crime?

Well this cuts in both ways. I mean, having a compliance program is both important having ex ante, that is before the crime is committed, because in this way you demonstrate that the crime has been committed by one employee of your corporation, but you have adopted all the needed preventive measures, but you can also adopt after the crime has been committed, so in this way you can demonstrate that you have adopted all the measures needed to repair, such as affronting payment to the victims or collaborating, cooperating with the prosecutors authorities. In this way, you can demonstrate to be a good corporate and obtaining lower fines, or be even absolved during the trial. It is interesting, this approach has been adopted by all the anti-corruption tools, all around the world, and there is no a great matter about which kind of responsibility the corporation is facing with. It’s no matter if it’s called criminal responsibility, rather than administrative responsibility, or something different. What is important is that all the anti-corruption legal tools take corporation as accountable for the crime committed, so all the corporations all around the world need compliance programs to prevent, or to repair, what has been done by its employees.

Okay, but what legal systems will actually demand the compliance programs? Is it just the USA, or do other jurisdictions also demand that?

No, lots of jurisdictions demand that. In Europe, it’s the case of Germany, Italy, United Kingdom, as I already told you. What’s important, and it’s really interesting is that the United States government has gone beyond. Interestingly they adopted compliance programs even in public procurement law, because through compliance programs, the United States government can select which kind of corporations it wants to deal with. The government use compliance programs as a benchmark for corporation reliability in government contracts. This has happened since 2008, the compliance programs have been implemented and required for federal contractors within the general legal framework for federal contracts in the United States. This is called responsibility determination. It’s a way to testify the integrity of the contractor, and for justifying this integrity, the government requires that a contractor has to have compliance programs.

So how is the assessment done? So you say it’s used by the federal government to assess a contractor.

Well, it’s done before getting the award, the Government looks if the corporation has implemented an effective compliance programs. The Federal Acquisition Regulations that are the general legal framework for this kind of government purchasing provides that before the award is done, the compliance program has to have implemented within itself a code of conduct, a training program, and especially such kind of measures that can prevent corrupting activities from the corporation in governmental contracts. So it’s the last stand that the awarding authority does before making the award. So in the United States the perspective is before evaluating the tender and just at the very end evaluating the contractor, because it’s a kind of cost-effective perspective. In this way, they are just going to give the award to really responsible contractors, who have implemented within themselves all the needed measures to prevent corruption.

So why don’t they do that at, let’s say, selection stage, like we have here in Europe?

This is a really interesting topic, and the legal thinkers from Europe and United States really discuss and argue about this. From their perspective, they do at the very end, because it could be perceived as an anti-competitive issue, stating who is a good contractor before and so will limit the competition if they will do this at the very beginning of the award selection, as we do in Europe. Another point could be that, by the United States government point of view, they think that in this way they can select the good contractor at the very end, and so it is really cost-effective because they just can pass to another contractor if the first one has not implemented the compliance program. It’s a really different perspective. I think the United States mood is not really the most objective way to award the contract, but at the very end it really complies with the law and ethics, because these compliance programs are really strict and the Governmental Accountability Office really takes care about integrity from governmental contractors.

That is the view in the US. You said before that the UK, and also Italy, they do have compliance programs. How do they compare with the American one?

Well, the implementation in Italy and in the UK have been really different, and I have chosen these two countries because they are both EU countries, but it’s a good way to show how it’s difficult and different implementing legal tools from a system to another one. Italy has adopted compliance programs in 2001 for regulating its corporate criminal liability, but the Italian implementation of the model has substantially failed, because several reasons. The legal tools for supporting the model, such as the code of conduct for the training programs, have been poorly realized in the Italian framework, and many Italian corporations have just copied the standard compliance and ethics program without adapting them to their specific needs, and last, the list of criminal conducts that the corporations have to avoid is really too extended. As a consequence, corporations cannot really focus on the white-collar crimes to prevent, such as corruption, and so the deterrence effect is really low in Italy. Lastly, the model has not been used by awarding authorities to stimulate governmental contractors’ reliability. They have been totally ineffective in promoting the integrity in public procurement.

So in your view, what should have been changed in the transposition from the USA to Italy?

Well I think what we should do is linking more criminal law, public procurement law and administrative law - this would be really a good step in promoting integrity in public contracts, if each kind of contractor will be proven and tested by its integrity within the supply chain, and with its commitment to integrity.

It’s really notorious that all the Italian corporations really have great problems with managing integrity within themselves, and training its employees to towards ethics and compliance with the law. By a legal point of view, as I say, the Italian model is really complicated because it is referred to such a kind of not-criminal liability of the corporation, because there is, after all, the criminal law principles that a corporation cannot be sanctioned by criminal sanctions, but must be sanctioned otherwise. This really shows that the legal principles can hinder the effectiveness of compliance programs. As I see, otherwise, as we were mentioning, the UK government has implemented the same tool in a really effective way, because they did not stick on with really difficult transposition of such kind of corporate liability. They have just set up a new, completely new criminal law provision that is the “failure to prevent bribery”. If a crime is committed within a corporation, the corporation is sanctioned just for having failed to prevent a crime, such as bribery, within themselves. This has been done within the Bribery Act of 2010 that has been considered the most advanced legal tool in fighting corruption, and the United Kingdom government have done really well, because it has also stimulate the corporation to commit themselves towards integrity, for example promoting the whistleblowing against other corporations or promoting the self-reporting during criminal trial. All this stuff does not exist in Italy, because we move in another, totally different criminal law context, which does not allow this kind of cooperation among corporation and public authorities. The overall approach in Italy is really bureaucratic, it’s really rigid, while in the United Kingdom, as I’ve seen, as I’ve studied, it’s really flexible and proactive, and it really helps to ensure integrity within the public and private relationships.

My question will then be, if that is the case, how can you be sure that the UK model would work in Italy well, when the American one didn’t?

 This is a really good question. Well, I think that the path, which at this point will be operating at the EU level, should be settled while implementing the new EU Directive on public procurement. We know that corruption is a really important point, as everybody has noticed, within the directive, because as we say at Article 57, we see corruption as a ground for exclusion, it could be considered corruption according to the national law of the tenderer, rather than the national law of the awarding authority, rather than the EU directive, the notion of corruption in directives on internal security, or according to EU convention against corruption. So the focus on corruption is really high. As I see, Article 57 could represent a step for all the awarding authorities in all Europe to evaluate corruption. So commitment to integrity for the corporations, no matter what the legal framework is at the basis of the national legal system. By operating at the EU-level the awarding authority, they can decide themselves if the corporation has effectively implemented a compliance program, if they have effectively committed towards integrity.

In your view, what should change at the EU level?

Well at the EU-level I think that it could be and should be improved the function of compliance and ethics program. Article 57 of the new EU directive really provides the possibility for the corporation and for the awardee to adopt the self-cleaning measures, but these self-cleaning measures have just been viewed until right now as a general provision that could be considered as an exception to exclusion, but cannot really push a corporation to act with integrity and complying with the law. I think Article 57 is an interesting starting point, but should be really implemented by the national legal system, giving more broad scope to compliance programs and where it is possible being required to the government a contractor as actually happens within the United States legal system.

Awarding authorities at the EU-level have a real important task right now. Being able to evaluate if a contractor could get the award because it complies with integrity and at the same time not being too much strict in evaluating the corporation if it’s not complying with a traditional scheme of compliance and ethics programs, because we know that these awarding authorities operate at a really low level, so they have not the same human resources or technical capacities to debar a company as happens in the United States legal framework. So it’s a really hard task, but I think the commitment to self-cleaning within the compliance and ethics program is the mood needed to merge together awarding authorities and corporations towards integrity within public procurement.

Do you think that should be then at the contracting authority level or at the member state level, or even at EU level? Because it’s one thing for you to, let’s say, block a company that has been accused of or been found involved in bribery actions as a contracting authority. It’s something different for the company to be sanctioned at the member state level or even at EU level.

Well, as I see, I think that what we can call the responsibility determination, I mean, verify if a contractor is good for getting the award, could be done at the awarding authority level. So each kind of awarding authority could be done for itself. Usually the awarding authority is the authority who knew more about the contractor, so should have the needed discretion for giving an award or not, according to Article 57, to the grounds of exclusion. What shall be instead done at the Member State level, rather than at the EU level, I think should be the debarment or the suspension of the corporation. In this case, a company, the corporation that has been debarred from an awarding authority, from a single Member State state, should be excluded from all the contracts from all the European Union awarding authorities. As I said, this is the same way they have done in the United States legal system. If a company has been debarred, let’s say, at the federal level, the same company cannot get an award at the state level. The debarment is automatic. So, as I see, we have started doing responsible assessment and it should be done at the awarding authority level without prejudice to the corporation, while a serious administrative decision as debarment it should be done at a higher level, maybe from a higher authority which should have extraterritorial jurisdiction, I mean operating in all the European Member States.

Very well. Speaking still on corruption, what should happen to contracts where corruption is found? Public contracts? 

Yes, sure. This is another topic I’ve dealt with during my PhD thesis, and starting from the United States legal system that has been my main reference, it’s really interesting noting that within the United States legal framework, the awarding authority has a really large discretion to terminate the contract if it’s proven that there awardee obtained the same contract due to corruption. There is no need for a criminal conviction for terminating the contract. It’s enough having a decision from the same awarding authority that certifies that the corruption has happened within the award. Nonetheless it is interesting noting that in many cases, especially in the most expensive contracts, even in front of corruption, awarding authorities prefer not to terminate the contract and going along with the original, even if criminal, awardee. This is the notorious case of Boeing, the case is also known as the Darleene Druyun Debacle, wherein a public official has manipulated the score in favour of Boeing and the company got the award. Well even in front of corruption, the needs of the military sector, in this case, the overall interest of the government, always prevail, and so the awarding authority preferred to continue the contract and not to terminate it, and continue with the previous awardee. I would like to underline that this happens also in Italy and also in other European Union Member States. The public interest to fulfil the contract always prevails. This also happens within the Expo case, that is a big case of public procurement corruption in Italy.

Which Expo? The Expo in Milan?

Exactly. Within Expo event in 2015 there was a big corruption scandal about the award of the public works for building the main pavilion, the main infrastructure in Italy, and even if it was possible to terminate the contract since they awardee had corrupted the award commissioned for such kind of public works. The supreme admissive judge in Italy decided the contract was to put on a compulsory administration that means that there is a legal expert coming from the State that managed the award, but the contract had not been terminated because the governmental authorities prefer not, they never want to terminate the contract, and this at the very end penalised the honest contractor because even in front of big corruption, they never get the award.

There are two different things here. One is a public interest in getting the job done, and I suspect that more often than not the contracting authorities don’t really care about the corruption, they just want to get the job done, because at the end of the day, that’s where their interests lie and also that’s where they’re exposed in terms of reputational risk. Whereas the corruption, it’s almost like, okay, so they’re corrupt but they’re still doing the job, so you need the job done. So it’s probably underplayed by the contracting authorities, in the decision makers’ heads, in their own framed set of values.

I absolutely agree with you, exactly.

So how can we change this?

Well, I have similar view to this regard. I mean, what could be done and what should be changed in this way it is preferring to terminate the contract and going on with another award. There are several contractual tools, they are called by Transparency International integrity pacts. Through these paths, adopted within the award, it is possible to provide that if the awardee has corrupted they award commission, and so the overall award has been corrupted. It’s possible to instantly terminate the contract and get the contract to the second economic operator within the award or to the other competing operators.

The thing is, it’s easy to do that if you catch the corruption, or you find the corruption and you’re that certain about it, which is not always easy, but if you do it straight away after you awarded the contract. If you’re six months or a year into the contract, you simply may not have the ability to go back to the second contractor because they may no longer be interested or have the resources to undertake that contract.

I totally agree. I think in this point is you really need better communication between criminal law and procurement law, because as we see in Article 57, for instance, we always need a final judgement about criminal conviction of corruption, and we know, at least in Italy, this really takes a lot of time, providing a final criminal conviction of corruption. What should be done, as it’s done in the United States, it is just relying on an initial decision by the awarding authority that something has gone wrong, something has been corrupted within the award, and so not waiting for the criminal conviction but evaluating if the integrity of the awardee is questionable, this could be done even at the EU level according to Article 57. It provides that a tender may be excluded where its integrity is questionable. I think this could be done also once the contract is awarded, and if the integrity of the awardee is questionable, terminate the contract as it’s possible according to Article 73 of the same directive, and so letting the second economic operator scored get the award.

Again, that still depends on a very quick finding that the tender is questionable and the tender is subject to those problems.

Absolutely, and I think this is far more better than waiting for a criminal conviction. It’s always a the decision that the awarding authority should take timely and should take, of course, with hearings and notice and comments from the awardee. Well the other path is waiting until the contract is done, discovering after that the corruption has been made, and then giving relief to the second economic operator, and this is going not to promote integrity with public procurement, but it is going to be an economical and a financial loss to the awarding authority: also to give relief to the second economic operator. So the damage is doubled at the very end. You have paid a corrupted contractor and then later you have paid an honest contractor because he didn’t get the award that he had deserved, by the way.

Very well. I think we are up to the limit of our time. Thank you very much, Franco.

 Thank you very much, Pedro, it was really interesting and thank you for your time.


#8 - Marta Andrecka (Aarhus University)

Framework agreements in public procurement

Interview with Dr. Marta Andrecka, Postdoctoral Fellow at the Department of Law at the University of Aarhus, where she is currently working on a project called "Dealing with legal loopholes and uncertainties within EU public procurement law regarding framework agreements" sponsored by the Carlsberg Foundation. Before taking up her present position she carried out her doctoral research in Denmark and Australia and worked at law offices in both Poland and the UK. The topic for today’s talk is framework agreements, a relatively popular way of undertaking public procurement.