#25 - Ana Cristina Calderon Ramirez (Maastricht Graduate School of Governance)

Regulating public procurement in Latin America and the Caribbean


Interview with Ana Cristina Calderon Ramirez from the Maastricht Graduate School of Governance on the regulation of public procurement in Latin America and the Caribbean. Ana is currently studying in-depth the creation and establishment of public procurement regulatory and monitoring entities in Latin America. She co-authored a paper entitled ‘Elements of public procurement reform and their effect on the public sector in Latin America and the Caribbean’, which was selected for a Best Paper Award in the International Public Procurement Conference 7 held in Bali in August 2016.


Hello Ana, welcome to the programme.

Hello. Thank you so much for inviting me.


Let's start by talking about your research in the paper that you’ve published with your colleagues. What is the background behind that paper, so what led you to actually write it?

Well I must say first that the story of the paper started with the research in which I have been working as part of the PhD studies at the School of Governance in Maastricht University as you mentioned before. When I was starting to explore the topics that I wanted to study, it appeared very interesting to analyse the elements of the procurement reform in Latin America and the Caribbean region, including the improvements of regulatory frameworks, the introduction of electronic government procurement tools, and the creation of specialised regulatory and monitoring procurement entities. In doing that I was looking for data, quantitative data to know and to understand a little bit more  about the historic  patterns of the reform and when that happened one of my colleagues and friends, Jorge Muñoz, who is the other author of the paper, and that happens to be econometrist specialised in public sector, told me about the PRODEV evaluation dataset that is compiled by the Inter-American Development Bank and that compiles the indicators related to the advances in public management in the region. So we started with him to check if there were any patterns or relationships in the reforms that have been happening and improvements with other indicators or variables including public expenditure or in institutional strengthening.

A year ago we found out that the creation of those procurement agencies was very relevant during the period of 2007 and 2013. Then we were talking with the other author, Leslie Harper, who is a well-recognised Fiscal Specialist at the bank, and with Jorge and Leslie we started to have conversations after work thinking what we can do with these indicators to meet the demands of our client countries. We have been asked for more information on these issues and we wanted to fill a  gap that is in the literature. So we decided to write a paper for the Procurement Conference number 7, , looking for the relationships or effects that the improvements of public procurement system have in the  perception of in our region. So I should say that this is totally a team effort that combined policy and research.


Sure. It makes perfect sense. But you got the idea to do that paper based on your work at the bank or based on your research, academic research that you’re doing for your PhD?

Well it’s a mix, you know, because when I started the PhD as I mentioned it I started to look for these datasets, we found some interesting things and then talking to Leslie who was my supervisor and Jorge,  I said “well we have so many clients needing information, policy information, we should do something with this” and she said “great”. So it’s when you find two ways that they meet together, you know, policy and research, so it was the two things.


Talking about the research that you’ve done in a little bit more detail. Why is it important and what aims were you trying to find out once you started working on it?

Pedro I should say that from an academic perspective, as we know much of the research in this field concerns itself with description, others with explanation of public procurement reform process. In Latin American, Caribbean region more than twenty years have passed since the reform began and since then there have been great strides in the modernisation and reform of the public procurement systems. These advances have included, as I mentioned before the creation of e-procurement platforms, the establishment of more comprehensive legal frameworks, the professionalisation of procurement officers, the use of procurement to promote social objectives, and inclusion of the green criteria in tendering opportunities.

Public procurement reform in the region has been analysed in issues of the impetus of the reform, the issues and constraints affecting these reforms, partial evaluations on specific country cases. However, if you look in the literature little has been written to identify and analyse the outcomes of these reforms and the impact that these reforms have had on the perception of public sector performance. So with this paper we wanted to address those gaps in knowledge by providing a conceptual framework for understanding the context under which public procurement reforms have been carried out and examining what we have called the first generation public procurement reform elements and the impact of those elements on public sector performance.


Those are the improvement in regulatory systems, electronic procurement and monitoring entities?

The three of them. The creation of electronic, yeah, the law and the institutional aspect, yes. Going back to the point that you mentioned before about the policy, I think we achieved four main things that are good for our colleagues in the international development field because first we were able to provide a framework that connects public procurement within public sector literature, policy literature, although these things very obvious in the public procurement has been an isolated topic, a ghetto topic. Sometimes we know that it is part of public sector administration but there is a lack of understanding of its importance and the role that it plays. We have seen public procurement reforms as part of huge reforms, public financial management reforms or anticorruption and transparency reforms, but not as a field that has its own space. So now it’s more like it has its own agenda, so we add that into the map of public sector reform and the elements that are included in that.

Second, we made an effort to clarify the effects of these reforms, both in Latin America and the Caribbean because most of the literature and policy literature covers Latin America but not the Caribbean. So here we use the dataset, the complete dataset with Caribbean countries and this is very interesting. And second, we tried to make clear what was first and second generation reforms and effects of these issues in other important sectors. Third, we demonstrated that although there is not much public procurement data as there is in other subjects, we can use real data, especially that information that has been collected by agencies, international development organisations and the countries itself. So sure we need more data and this has been an issue for so many years but now we have data and we have to work with that data so let’s start to use what we have.

And finally, fourth there was something very important to us, this is a research with a policy perspective as well. Because sometimes you heard that research is research, full stop. If it’s used that’s another stuff, however we tried to combine policy and the academic world in this paper. For the people who are listening to this podcast I want to let you know that we have a big network on government procurement in the region that is called the INGP and we have so many conference as part of these network and so many trainings, and there we have heard so many people telling us that the research is very hard to understand and other papers are very light. So we wanted to combine that both academics and policymakers could enjoy and use what we wrote and more important what we found.


I think that’s a very important skill. One of the things I don’t like in academia in general, and I used to be a lawyer before I became an academic, is that so many of our colleagues write for themselves and their colleagues as well and they don’t understand or they don’t make an effort to actually write aiming to generate impact or at least so that their research can be understood and digested outside academia.

That’s great that you say that because that’s what we are doing now and that’s what I’m doing as part of this PhD, that is very different programme because it’s for policy people, so that helps a lot.


So you’re looking at some indicators and the effects and the impact that certain reforms in public procurement had had in the Latin America and also the Caribbean. What were your main findings?

I have here to make a clear point. These findings are for a dataset that was between 2007 and 2013, so let’s make that clear first. On that regard, based on the three main areas of the procurement reform evaluated we find out that the creation of a procurement agency had the largest impact on the development of public procurement systems between those years. Indeed the increase of the scoring of public procurement system in the countries where a public procurement agency was created was about 0.5 points above the scoring of the public procurement system in those countries where an agency was not created.

Second, there is a link between the creation of a public procurement agency and other variables related to the perception of public sector performance and the competiveness of the countries as well.


So bearing in mind that you didn’t analyse in detail the characteristics of each public procurement agency because they may be different in different countries, so that is just a general…?


Yes. But as a part of the dataset you have that, you have the different characteristics so some of them are not agency, some of them are just entities or  a department within a ministry. So the dataset includes that.

The effect of creating a new procurement agency were also statistically significant to explain improvements in variables such as favouritism in decisions of government officials and transparency of government policymaking, the government procurement of advanced technological products and their competitive index among others. For example, with the regard to the perception of favouritism in the decisions of government officials, the creation of an agency may have helped to improve this indicator by 2.6% as a consequence of its monitoring role, ensuring that procurement processes are carried out in accordance with the country’s legal and regulatory framework in a fair and impartial manner. Additionally, by lowering the cost for entry, the procurement processes become more competitive and less likely to be perceived as biased towards a particular individual or firm. And the creation of an agency had a gross impact of 9.9% on improving transparency of government policymaking. This effect might be explained by the development and monitoring of technological tools, electronic platforms for example, when public procurement agencies are fully functional and compliant with transparency regulation.

Second, there is a link between the creation of a public procurement agency and other variables relating to the perception of public sector performance, as you can see in the paper, and the competitiveness of the countries as well. We found out that the effects of creating a new procurement agency were also statistically significant to explain improvements in variables such as favouritism in the decisions of government officials, transparency of government policymaking, the government procurement of advanced technological products and the competitive index among others. For example, with regard to the perception of favouritism in the decisions of government officials, the creation of an agency may have helped to improve this indicator by 2.6% as a consequence of its monitoring role, ensuring that procurement processes are carried out in accordance with the country’s legal and regulatory framework in a fair and impartial manner. In addition by lowering the cost for entry, the procurement processes become more competitive and less likely to be perceived as biased towards a particular individual or firm. Additionally the creation of an agency had a gross impact on, let me read this, of 9.9% on improving transparency of government policymaking. This effect might be explained by the development and monitoring of technological tools such as the e-procurement platforms or e-GP systems, when public procurement agencies are fully functional and compliant with transparency regulations. Also the creation of an agency shows an impact on the burden of regulation as it brings more transparency and competition when a procurement process is planned. Because of its strategic vision and understanding of the issues that are happening in the country and the needs that the country has, the procurement regulatory agency is able to develop a clear and appropriate regulation increasing private sector trust in public procurement and generating increased participation in the public procurement marketplace.

And third, the most important part I guess is that having a public procurement agency that is fully functional is absolutely critical for the reform process itself, at least in our region that has been a very important issue because an agency that is responsible for policy and monitoring is needed to balance the competing goals that the country has, whether it’s fighting corruption, promoting transparency or trying to achieve value for money effectiveness and efficiency. An office with just administrative functions and no strategic mandate, focusing only on transactions, is not going to be able or to have the incentives to advocate for deep and sustained reforms. In our region legislative and technological reforms have been very, very important but without an agency that can provide a strategic direction, leadership ambition to leverage those changes in a holistic fashion, the reforms are going to stall or be implemented in a disjointed and piecemeal way.


Can you give us examples of those agencies?

Yes, sure. For example  Guyana. Guyana has a great law and you can call Caroline Nicholas about this to ask her. She has been there for a while and she has help us in the region for our regulatory frameworks, you cannot imagine how much. But they had a beautiful law as we said in Spanish, a beautiful law, but the implementation has been very, very hard. They started last year to have a procurement entity to be responsible of the implementation of the law, bringing more transparency by having and monitoring the e-procurement system and this has been helping a lot. The same in Colombia. Colombia’s a great example and now Colombia is very fashionable because, we are trying to modernise the public sector. The agency is new, it was created in 2011 but it started in 2012, and these years have been very important for the country in the public procurement system because this agency has given to the public procurement system impetus, you know what I mean.


Yes, impetus.

Now people recognise this public procurement, they understand, they know that they can take and bring to the public marketplace, they can offer, they can buy, it’s easy to understand. So that helps a lot. The agency helps a lot.


Very well. Still on the main findings part of our conversation, I found interesting in your paper that you think the regulatory regime and the legal framework is not as important to improve practice or to achieve the outcomes that it’s supposed to achieve than in comparison with the procurement agencies?

That’s the reason why when I started to talk and said “please we should be very careful on this, this data is between 2007 and 2013”. Before 2007 I am sure if we go before that we could find that the legislative and the legal framework were very important, were the most important bit among those other factors. The thing is that in 2007 most of the countries already had the legal framework established but not implemented.


Okay, that makes sense.

So 2007 and 2013 was like an explosion of these entities and I explained in more depth in my dissertation that I call “agencification” . I think we should make this point clear.


So in fact you’re saying is that without the law and the regulatory framework already existing it would be hard for the procurement agencies to actually achieve anything?

Of course. Of course. And more in our region that’s very important. No, no, no, we should say that.  I mean, you cannot stay forever with a perfect regulatory system but no-one complies with that regulatory framework, you know?


Yes I know. I know the feeling. Very well, okay. So those are the main findings of your paper. You’ve been very cautious saying that okay this is only, you only analysed the data between 2007 and 2013 but can we extract wider application of the main findings? Do you think that they would be valid or would be useful for other areas of the world like Africa or East Asia?

Well I think with a good dataset we could do something with Africa and Asia. And as part of my dissertation, not as part of this paper, I have been having some interviews with a specialist at the African Development Bank and the Asian Development Bank and World Bank. But it seems that they are very different in the form that they are building up their systems, when I say system there’s no electronic system but the complete public procurement, system. For example, isomorphic pressures are different, the way of the path dependency is very different in those regions of the world. However, if we can find good data maybe we can find the same things, agencies will help to boost the efforts of the reform, but I wanted to be very careful on that point.


Your paperwas selected as one of the best papers in the International Public Procurement Conference that happened in Bali and that’s, it’s a great achievement for you to have been able to do that as a PhD student. What else are you doing now at this moment in time in term of research? So you looked at Latin America, what else are you doing for your PhD?

Well I’m not doing anything regarding the Caribbean right now. Maybe I will write a chapter for a book that is going to happen in 2017 with Inter-American Development Bank . Nowadays I’m just focusing on the public procurement agencies, not just the departments or small units but the agencies in Latin America. But I wanted to check the isomorphic pressures and the policy transfer pressures in the process when you establish these type of agencies. So that’s what I have been doing right now. However, I think I want to mention something if you don’t mind?



Because we have been doing some other stuff and I think the wider applications of these main findings could be from different sides. So from the country side I’m sure when the conference of the regional network and public procurement that is going to happen in November in Jamaica I’m sure the countries will talk about these issues and I am really sure that those countries that do not have these type of entities can learn a lot on their benefits and their impact, not just for the procurement system but for the public sector as well so they can sell more easily these type of reforms to their ministers. And others that have the agency on paper will understand the importance of the implementation because sometimes they have it there in the law but they haven’t had the time, the money or the willingness to make it real. For the donors it shows how the reform has succeeded in the region,  how much of the money that have been provided for the strengthening of procurement system is having a real effect. Actually in the paper you will see that we found how was improvement related to each dollar that was provided by the Inter-American Development Bank, so that was a good one for us. And it provides us an snapshot on what is happening in the sector, so that’s good for them to know how and where we are right now.  So I think that’s the wider application of the paper.


One final quick question to wrap up the interview. What future steps are you going to take in terms of research?

First keep collating data. Yes, because we need to collect data and we are doing this in the countries, donors. Maybe you can have another podcast on the methodology for the assessment of public procurement system that we have been working with the OECD. They have it right now for discussions in its website.


So they are reviewing it at the moment?

Yeah. They’re reviewing it. We were part of a taskforce that reviewed the tool, so that would be very important to us because for developing countries we use that to monitor the reform. The country data as well, the initiative of open contracting and, or open data have been helping a lot in this task, so we will keep collecting data.

Second, more research, more focused research, maybe about each innovation or change, especially e-GPP or green procurement, new entities or so forth. I know right now and. We should have more research on the quantitative impact of the reforms because mostly the literature that we have is anecdotal. Probably the analysis of the second generation reforms in public procurement, green criteria in tendering.

Third future step is to complete my dissertation work, crossing my fingers , I think it’s time.


That should be the first one I would say!

Oh my god! But here I wanted to talk about the paper I mean I was trying to be fair and say “well we’ll be collecting data, we will do more research” but yeah, you are totally right for me, it’s completing my dissertation. And  when I finish I will let you know about that for sure.


I think that’s a great way to finish the podcast. Ana thank you very much for coming.

Oh thank you so much for this. I really appreciate it that you inviting us.

#19 - Stephan Litschig (Graduate Institute for Policy Studies)

Does an increased audit risk affect corruption practices?


Interview with Stephan Litschig, Associate Professor at the Graduate Institute for Policy Studies, GRIPS, in Tokyo. His research interests are in the area of development, public and political economics. His work has been published in the American Economic Journal: Applied Economics, the Journal of Public Economics and the Journal of Human Resources. He also serves as an Associate Editor of the Journal of Human Resources. Today’s talk is focused on his joint paper with Yves Zamboni called ‘Audit risk and rent extraction, evidence from a randomized evaluation in Brazil’.




I find your paper with Yves Zamboni very interesting and what I would like to know for us to start the conversation with was what led you to carry out this research.

The motivation was that we know that let’s say anecdotally, corruption is very prevalent in many countries but it is much less known what the people could do to reduce corruption. And by corruption what I have in mind in particular is in public procurement for example that public officials will take kickbacks in return for steering the contract to certain suppliers but I also have in mind a broader concept of corruption which is rents in the delivery of public services where the fact is that in many countries public servants don’t show up for work, teachers for example or health service providers. So the basic motivation is just that we know that procurement and service delivery in general could be better and then the question was can we change this by increasing the scrutiny that comes from the top basically, increased audit risk.


And why doing this in Brazil specifically?

So this was an opportunity I had come across the fact that in Brazil they have this peculiar way of auditing local governments which is done at random so there’s literally a lottery that is done that’s actually on television and so that, what that gives you is that it gives you comparable data on audit findings across Brazil, so for a large number of municipalities, and then I basically got in touch with this agency and wrote to them and proposed this and my current co-author was the one who picked it up and said “look, this is interesting, let’s do this together”. So it’s basically the fact that in Brazil they were doing a particular interesting way to conduct these audits and what we did with Yves we designed this study on top of this existing lottery where we added another layer that basically put aside a hundred and twenty municipalities during one year into a high audit risk group so these guys knew that, basically in the control group audit risk per year was something like 5% but if you were in this high audit risk group you had like a 25% chance to be audited over the subsequent year.


How was that selection done? Was that selection done by you or specifically was that a purposeful sampling or was it done at random saying “you guys in this year actually have a higher audit risk”?

Yeah, exactly. So it was purposefully at random if you want so, you know, they were already doing this random sampling using a literal lottery, like it’s the same machinery that was used for a money lottery in Brazil, that was very popular so they used the same machines, then we also used those machines to set aside a group of municipalities that, because it’s a random sample should have all observable and unobserved characteristics the same as those that were not selected. So then the idea is that if later on once the auditors go in and they find differences in the incidence of irregularities of matters of corruption this has to be because of the only difference between the two groups which is that they were exposed to ex ante higher audit risk.


And did they know that in advance, so were they informed?

Yes. So that’s an excellent point. So the overall lottery that was regularly done by this audit agency CGU was very well publicized so in that sense we could assume that they knew about it but to make sure that those that were put into this extra scrutiny high audit risk group, that they were aware of that, in addition CGU also sent them a letter and said “by the way, for one year you’ve been selected, you are one out of a hundred and twenty municipalities and at the end of the year we’ll take thirty of you to be audited according to our usual audit procedures”. So they did also receive a letter.


In terms of communication of information to those municipalities, so you guys sent a letter but if you’re looking into corruption to the municipality you’re looking into probably I would suspect multiple levels within the organisation, so did you expect the information about the higher audit risk to filter from the top down or just for that information to remain with the person or the persons that received it?

So we were agnostic about that, so the idea was that perhaps, so the mayor for sure would receive the letter…


Sure, yeah.

…but then some of the people in his cabinet would receive it, then, whether he would communicate this to, down to the health offices and all the teachers we were sort of, we didn’t want to micromanage that so we just said “okay, let’s just send this letter”. The key thing that changed was that objectively they were in a high audit risk group so we thought about this but we thought we don’t want to make this too artificial in a sense. Suppose the government would increase the audit intensity, right, they’re not going to send around lots of letters, they’re just going to do it, they might inform some people but we wanted to keep this as real as possible. But it’s true so what that means is that we don’t know how far the information travelled, that is a limitation.


In terms of findings, what were your main findings?

So we find that when we look at procurement outcomes we find that there is a reduction in both the share of the amount of audited resources that has evidence of corruption, so both in money terms but also when we look at it at the procurement process level and there’s something like 1,600 procurement processes that we have the data on and we find that at that level as well there’s a reduction in the probability that auditors find evidence of corruption. Now what does corruption mean? We basically use, so these auditor reports have been used before in the literature so we use existing corruption codings and one unambiguous example would be the auditors go in and they find that the firm that, you know, received the contract in fact doesn’t exist. So that [Pedro laughs], it’s scary but these things happen, you know. Other…


I know.

…other things that, right. Yeah, yeah, well, you know, some people are very surprised but I guess you’re more of an expert in this, in this area. But yeah, so other things are when they go in, the auditors go in and they find that the government paid more than what they had agreed to in the auction basically, without gaining any other compensation.


I was laughing because a few years ago in Portugal when we started collating the contract data, some people that I know in Portugal they cross-referenced data from that particular database, the contract database with information from the companies registry and they found that certain companies were being awarded contracts even before they existed. That’s why I laughed because I’ve seen that kind of stuff happening before!



So it’s not only Brazil, yeah.

No, I’m pretty sure. You know, so this, you know, I’m getting ahead of myself here but I think in terms of future research there is a lot to be done here, just sort of, just in terms of measuring how frequent does that happen in different countries in different contexts, I think we just know very little about that. And, you know, one reason for that that we know little about it is obviously data, data problems of merging different datasets but the other one is that in most countries the audit agencies would target sectors or firms where they think there’s a high likelihood of corruption and you end up with a very unrepresentative sample of things of what’s actually, what the state is in the overall country. The nice thing about Brazil is that because of this random sampling what we have here is basically a, you can think of it as a random sample so if we find for example that the incidence of corruption is something like let’s say 40% amongst restricted modalities, that’s one of our main findings, we can sort of confidently say that’s more or less what’s going on in the public sector at a local level in Brazil, as opposed to, you know, this reflects the tip of the iceberg or something like that. But, you know, to do that you need the random sampling and that’s, no other country in the world does that so if you want to do comparable studies across, just, you know, compare the incidence of corruption across countries you have to convince public officials to do things in a random way and that’s often very tough. But that’s, in terms of future research that’s sort of where I think this could go.


That’s one of the point that I would like to drill down a little bit further which is in terms of processes, which processes have you found out that were more likely to generate corruption or have corruption arise from them?

Yeah, so that was an interesting find is that for the first time in our study we have the detailed process level data which means that basically that process is it’s a given purchase, they vary a lot in terms of magnitude but what we know for each purchase is was it done through a modality that restricts competition in some way or not, which is an open auction and you specify in the call what you want and then anybody can pretty much apply. So we group modalities, there’s more than those two but we group them into two and the finding is pretty stark, is that, amongst let’s say public procurement auctions the incidence overall of corruption is much lower, something like 20% and it doesn’t change at all, whereas amongst modalities that restrict competition and give more discretion to the public official, the overall incidence in the control will be something like 40% and there we find the reduction, it only occurs there, something like 15 percentage points. So it’s only the modalities that restrict competition, that afford more discretion that, where this corruption seems to be going on and where it is affected by higher audit risk.


Does Brazil have financial thresholds in terms of determining which kind of modalities you’re going to use, if you’re going to use the auctions or you’re going to use the restrictive competition ones?



Or is it just done by case-by-case scenario?

No. There are thresholds and one of the things that the auditors look at and that we also, that is part of some of our corruption measures is in fact whether they basically fraction the purchase to avoid those thresholds. So there are thresholds.


Because that is fascinating, especially, or even in Europe because the financial thresholds in Europe are quite high so you only apply rules for contracts above 100 and something, for goods and services 100 and something thousand Euros if I’m not wrong or more, or even more if you’re a local authority. And I’ve always thought that it was not a very good idea, especially on corruption grounds but I’ve been writing about that area over the last few years and I’m going to revisit it again and you’ve just given me a new piece of the puzzle for me to work with which is actually where there is more discretion for public procurers we are creating the conditions to generate more corruption or at least make corruption more likely?

So that’s what our findings suggest for the first time that at least in Brazil in this particular context this discretion tends to be abused, okay. Now how externally valid is that? As you say I think that’s…


Yeah, we don’t know?

That’s research, we don’t know, that’s right.


Okay. But in that sense I’m writing about these ideas and obviously this area of public procurement from a legal perspective so I can afford the limited availability of data to try and shape my thoughts, although obviously I want it to be sustained by evidence and…?

Mm, and of course, of course.


Right, okay. So I’ve got a few extra comments for you.

Sorry Pedro, can I, because you asked and I’ve been going off topic I think so the paper is about procurement but if you don’t mind?


Yes, sure.

One of the things we also look at is this idea that when you increase the audit risk service providers are more likely to actually show up for work, okay. So there’s, it’s not corruption in terms of stealing but corruption in the sense of you’re paid to do a job and you don’t do it so, and the reason we can do that is because the CGU audits they include that as a standard procedure for them to basically also try to assess whether for example in health service delivery the officials, and these are going to be doctors and nurses but also going to be health workers that are supposed to visit the health facility users preventively, so the idea is that it’s cheaper to prevent than to cure but to, the prevention requires that these guys go out there and visit the families to find out whether someone is ill so that the prevention can actually take place and that’s costly of course for these agents. So what we looked at there or what the auditors looked at is they ask a random sample of potential service users in a given area they make household visits and they ask “are you receiving these visits from these preventative healthcare workers?” They also ask “when you go to the health post is it open when it should be open and when you go there is the doctor there?” These kind of questions so, and they ask some questions in control and treatment municipalities, okay. Now what we find on that front is that although in the control group there is not perfect performance in the sense that people do report that the health post is not always open as it should, the doctor is not always there, the increase in audit risk has no effect on that performance, on that compliance with your stipulated job requirements. In stark contrast to what we find in procurement, we find this reduction.

So what we, we were puzzled by this, it was we didn’t expect that but then what we do in the paper is we obtained a standard economic model and we see whether we can rationalise these findings. And one explanation for these results could be that when you get audited in procurement basically you can’t argue with those findings, you know. If the auditor finds that the firm that you awarded the contract to doesn’t exist there’s not much more you can say, whereas if the auditor goes in and he finds that people report that the doctors were not in the health centre when they should have been, the doctors are going to say “no, I was there” and the audit finding doesn’t generate that kind of hard evidence that would make people work. So we think that ex post we can rationalise our findings so the fact that health service providers might just not be worried by the audit because they can always say “well no, no, I was there” or “I was in another room” or “I was, I don’t know, I was visiting a family out in the field” so it just doesn’t generate the same kind of evidence. And so that, you know, what that, it just, I guess it also, it’s not, obviously it’s not clear how general the finding is but it raises this issue that the quality of audits can be useful to monitor certain things but not others. And then that has implications for audit design so perhaps the auditors can save these visits because at the end what’s being generated is not really useful that much.

Anyway, so I just wanted to mention that. I know this is a procurement audience but because the increased audit risk happened for everybody in the entire municipality and because the auditors look at these things as a matter of routine we thought that it’s useful to look at that as well, even though it’s a null result we think it’s interesting.


That is interesting and I was wondering as I was listening to you if it may be that the consequences are actually as far as I know very different in case you are found involved in corruption, actual corruption in a public procurement process or just not doing your work properly?

You’re exactly right. So there’s an additional reason which is that if you’re found stealing from any public procurement you can basically go to jail, whereas even if you’re convicted that you were shirking on the job you might lose your job at most.


At most, yeah.

You know, the sanctions are just less as well. So you’re exactly right, that will be another explanation. To be fair there’s also the explanation that as you said earlier, right, information might not have travelled to those guys, could be that the…


They did not know?

They didn’t know they were not aware of this audit thing and for them they were not treated in some sense. And, you know, we won’t know [laughter] so…


I’ve got another question here, a little bit more specific is that I think it’s on page 11 or 12 that you say that corruption or at least stealing less today does not imply that more can be stolen tomorrow, so you argue that because the federal funds cannot be saved for periods longer than the budget year that there’s no possibility that someone will not steal today so that they can steal more the next year because they’re just going to lose the budget?

Uh huh.


I agree with you, I think that is true but I was wondering if you found something within, any difference within the same budget year? Let me explain, what I’ve seen working with public procurement officials in different countries is that sometimes when we get close to the end of the budget year you still have some of your budget left and you need to spend it, you need to spend it and you don’t usually spend it in the best way possible, so that would increase the likelihood that someone would be tempted to take some money or be corrupted in one way or another. So what I’m wondering is if within the same budget year or fiscal year have you seen any differences or potential differences in terms of one is more, when is corruption more likely to occur?

Mm. It’s an excellent question but without data…


Without data, yes.

We are bit limited on that because we don’t know basically when the purchase was decided, you would need monthly data in some sense, right? You need to know when was, and that we just don’t have. I think for other countries, I think for Chile there’s actually a study that basically shows what you’re saying. Maybe you had that study in mind? But so …


No I don’t, I don’t. I would like to know more about it so if you know it just give me the information afterwards and I’ll look it up?

Yes. I don’t have the authors off the top of my head right now but this has been empirically documented. Now I think what they look at if I remember correctly is just, do they, so I’m not sure they actually have audit findings on these transactions, I’m not sure if they can say that it’s more corrupt if it’s spent towards the end of the year but things do happen that are special at the end of the year, that much I remember.


Okay, very well. One final question, next steps and future research plans in this area, do you have any? Do you think that you’re going to still revise the paper one more time or…?

Well the truth is that it’s not published yet so I’m sure I’ll revise it, that’s just, but that’s nature of the game and, you know, in fairness to the referees it usually gets better. But the paper is what it is. What I do find though is that academically people tend to, at least the referees I’ve got, they often say “well it’s not really surprising, we already know that an increase in audit risk will reduce corruption, it’s sort of obvious” and I think that’s just wrong and I think it’s actually, it hurts public policy because you just assume these things and you don’t study it. The fact is that we don’t, we know very little about the incidence of corruption in procurement and service delivery more generally across the world and then what works to reduce it, if it has worked in one context is it going to work in another one? We just have no idea. So I, there’s the, the short answer to this is that I think I’ll continue working on this topic trying to see whether more monitoring or other kind of monitoring or even also incentive schemes can improve service delivery and reduce corruption but I’ll have to find willing partners and that’s, you know, that is the, you have to work, to do this kind of stuff you need connections with governments and that’s, that is sort of the major constraint, okay. So I can say I want to do that, the question is going to be am I going to find governments that are willing to do that kind of stuff and I can just, all I can do is try. So that’s one.

Another area where this can go where it’s a little bit, a different question is the fact that public procurement has in recent years moved very much online so e-procurement where from a monitoring perspective what’s interesting about that is that there is much more information now about let’s say the awarding stage, right, so you can basically go online and see what kind of contracts are available for which you can bid, you can see afterwards, you know, who won, so there’s just much more transparency in the entire awarding stage and I have a project with two other co-authors where we try to see when, under this increased transparency in the awarding stage corruption or irregularities get shifted more towards the execution stage because in order to know whether the price agreed was actually the price paid or whether the quality was according to what was agreed or not, whether it was substandard quality in the execution, you actually, you still need an audit. This whole movement of e-procurement hasn’t changed the fact that we know very little about what’s going on in the execution stage and there’s our worry that just the focus of corruption might shift towards execution. So that’s just, you know, putting sort of a question out there that we’re working on but we haven’t, I don’t have any results but I think that’s, in the future that is interesting, So less or increased transparency at the awarding stage might actually shift, just shift things around and overall towards the execution and that, and overall you might not get much of a benefit. So that’s a worry that we have and we want to investigate that.


That’s a very good couple of points that you’ve made there for us to conclude the show on. I would just like to say that I agree with you that the more transparency that we have during the awarding stage, the more likely it is that whatever, let’s call it irregular behaviour or illegal behaviour taken by agents is going to move elsewhere. You are right in looking at the execution stage because execution stage unfortunately so far has not been subject to as much scrutiny as the award stage and that for me is a mistake. It’s a big, big mistake in terms of policy and I’ve been against that view of procurement, that procurement is only the award stage and not execution for many years. On the other hand something that you may not be aware of is that at least in Europe under the new EU Directives it’s now possible in variable terms unfortunately in my view for contracting authorities to have conversation with suppliers even before they open the award stage, so that means that there’s another, there’s another moment where corruption can occur which is actually before the award procedure. So even if you have let’s say an open procedure, so what you guys would call an open auction for example, if you have an open procedure, very transparent, very clear, you look at it, just at the procedure and you will not find any corruption but you don’t know that for example one of the bidders was told in advance what was going to be included there and for example shaped the actual award criteria.

Right, so the buyer can tailor the call to whatever they, and you’re saying this is legal in Europe?


What I’m saying is that it’s legal nowadays for the contracting authority to have conversations with suppliers before it launches a tender and the way that it is drafted in the directive it’s very, very liberal and way too much liberal for my taste.

Interesting. But so is this a change that’s happening?



Or are you, ah.


Yes, it is a change. I mean economic operators know but contracting authorities have been asking for this for many years, they say that if they are forced to use an open procedure, say an open auction, that is very difficult for them to know in advance what they really need and how to spec it out correctly so they need to talk with the market in advance, so if they talk with the market in advance the procurement is going to be better and I just say bollocks, you’re just creating the conditions for undue influence to occur. But that’s another area that is coming up at least as a potential area of research in corruption in public procurement.

Well very interesting. Thanks a lot for letting me know. Well, you know, let’s keep in touch. I mean clearly I have a lot to learn about what’s going on in Europe.


Okay. Thank you very much Stephan, was a pleasure talking with you.

Thanks Pedro. See you.