Brazil

#19 - Stephan Litschig (Graduate Institute for Policy Studies)

Does an increased audit risk affect corruption practices?

iTunes

Interview with Stephan Litschig, Associate Professor at the Graduate Institute for Policy Studies, GRIPS, in Tokyo. His research interests are in the area of development, public and political economics. His work has been published in the American Economic Journal: Applied Economics, the Journal of Public Economics and the Journal of Human Resources. He also serves as an Associate Editor of the Journal of Human Resources. Today’s talk is focused on his joint paper with Yves Zamboni called ‘Audit risk and rent extraction, evidence from a randomized evaluation in Brazil’.

 

Transcript

 

I find your paper with Yves Zamboni very interesting and what I would like to know for us to start the conversation with was what led you to carry out this research.

The motivation was that we know that let’s say anecdotally, corruption is very prevalent in many countries but it is much less known what the people could do to reduce corruption. And by corruption what I have in mind in particular is in public procurement for example that public officials will take kickbacks in return for steering the contract to certain suppliers but I also have in mind a broader concept of corruption which is rents in the delivery of public services where the fact is that in many countries public servants don’t show up for work, teachers for example or health service providers. So the basic motivation is just that we know that procurement and service delivery in general could be better and then the question was can we change this by increasing the scrutiny that comes from the top basically, increased audit risk.

 

And why doing this in Brazil specifically?

So this was an opportunity I had come across the fact that in Brazil they have this peculiar way of auditing local governments which is done at random so there’s literally a lottery that is done that’s actually on television and so that, what that gives you is that it gives you comparable data on audit findings across Brazil, so for a large number of municipalities, and then I basically got in touch with this agency and wrote to them and proposed this and my current co-author was the one who picked it up and said “look, this is interesting, let’s do this together”. So it’s basically the fact that in Brazil they were doing a particular interesting way to conduct these audits and what we did with Yves we designed this study on top of this existing lottery where we added another layer that basically put aside a hundred and twenty municipalities during one year into a high audit risk group so these guys knew that, basically in the control group audit risk per year was something like 5% but if you were in this high audit risk group you had like a 25% chance to be audited over the subsequent year.

 

How was that selection done? Was that selection done by you or specifically was that a purposeful sampling or was it done at random saying “you guys in this year actually have a higher audit risk”?

Yeah, exactly. So it was purposefully at random if you want so, you know, they were already doing this random sampling using a literal lottery, like it’s the same machinery that was used for a money lottery in Brazil, that was very popular so they used the same machines, then we also used those machines to set aside a group of municipalities that, because it’s a random sample should have all observable and unobserved characteristics the same as those that were not selected. So then the idea is that if later on once the auditors go in and they find differences in the incidence of irregularities of matters of corruption this has to be because of the only difference between the two groups which is that they were exposed to ex ante higher audit risk.

 

And did they know that in advance, so were they informed?

Yes. So that’s an excellent point. So the overall lottery that was regularly done by this audit agency CGU was very well publicized so in that sense we could assume that they knew about it but to make sure that those that were put into this extra scrutiny high audit risk group, that they were aware of that, in addition CGU also sent them a letter and said “by the way, for one year you’ve been selected, you are one out of a hundred and twenty municipalities and at the end of the year we’ll take thirty of you to be audited according to our usual audit procedures”. So they did also receive a letter.

 

In terms of communication of information to those municipalities, so you guys sent a letter but if you’re looking into corruption to the municipality you’re looking into probably I would suspect multiple levels within the organisation, so did you expect the information about the higher audit risk to filter from the top down or just for that information to remain with the person or the persons that received it?

So we were agnostic about that, so the idea was that perhaps, so the mayor for sure would receive the letter…

 

Sure, yeah.

…but then some of the people in his cabinet would receive it, then, whether he would communicate this to, down to the health offices and all the teachers we were sort of, we didn’t want to micromanage that so we just said “okay, let’s just send this letter”. The key thing that changed was that objectively they were in a high audit risk group so we thought about this but we thought we don’t want to make this too artificial in a sense. Suppose the government would increase the audit intensity, right, they’re not going to send around lots of letters, they’re just going to do it, they might inform some people but we wanted to keep this as real as possible. But it’s true so what that means is that we don’t know how far the information travelled, that is a limitation.

 

In terms of findings, what were your main findings?

So we find that when we look at procurement outcomes we find that there is a reduction in both the share of the amount of audited resources that has evidence of corruption, so both in money terms but also when we look at it at the procurement process level and there’s something like 1,600 procurement processes that we have the data on and we find that at that level as well there’s a reduction in the probability that auditors find evidence of corruption. Now what does corruption mean? We basically use, so these auditor reports have been used before in the literature so we use existing corruption codings and one unambiguous example would be the auditors go in and they find that the firm that, you know, received the contract in fact doesn’t exist. So that [Pedro laughs], it’s scary but these things happen, you know. Other…

 

I know.

…other things that, right. Yeah, yeah, well, you know, some people are very surprised but I guess you’re more of an expert in this, in this area. But yeah, so other things are when they go in, the auditors go in and they find that the government paid more than what they had agreed to in the auction basically, without gaining any other compensation.

 

I was laughing because a few years ago in Portugal when we started collating the contract data, some people that I know in Portugal they cross-referenced data from that particular database, the contract database with information from the companies registry and they found that certain companies were being awarded contracts even before they existed. That’s why I laughed because I’ve seen that kind of stuff happening before!

Excellent.

 

So it’s not only Brazil, yeah.

No, I’m pretty sure. You know, so this, you know, I’m getting ahead of myself here but I think in terms of future research there is a lot to be done here, just sort of, just in terms of measuring how frequent does that happen in different countries in different contexts, I think we just know very little about that. And, you know, one reason for that that we know little about it is obviously data, data problems of merging different datasets but the other one is that in most countries the audit agencies would target sectors or firms where they think there’s a high likelihood of corruption and you end up with a very unrepresentative sample of things of what’s actually, what the state is in the overall country. The nice thing about Brazil is that because of this random sampling what we have here is basically a, you can think of it as a random sample so if we find for example that the incidence of corruption is something like let’s say 40% amongst restricted modalities, that’s one of our main findings, we can sort of confidently say that’s more or less what’s going on in the public sector at a local level in Brazil, as opposed to, you know, this reflects the tip of the iceberg or something like that. But, you know, to do that you need the random sampling and that’s, no other country in the world does that so if you want to do comparable studies across, just, you know, compare the incidence of corruption across countries you have to convince public officials to do things in a random way and that’s often very tough. But that’s, in terms of future research that’s sort of where I think this could go.

 

That’s one of the point that I would like to drill down a little bit further which is in terms of processes, which processes have you found out that were more likely to generate corruption or have corruption arise from them?

Yeah, so that was an interesting find is that for the first time in our study we have the detailed process level data which means that basically that process is it’s a given purchase, they vary a lot in terms of magnitude but what we know for each purchase is was it done through a modality that restricts competition in some way or not, which is an open auction and you specify in the call what you want and then anybody can pretty much apply. So we group modalities, there’s more than those two but we group them into two and the finding is pretty stark, is that, amongst let’s say public procurement auctions the incidence overall of corruption is much lower, something like 20% and it doesn’t change at all, whereas amongst modalities that restrict competition and give more discretion to the public official, the overall incidence in the control will be something like 40% and there we find the reduction, it only occurs there, something like 15 percentage points. So it’s only the modalities that restrict competition, that afford more discretion that, where this corruption seems to be going on and where it is affected by higher audit risk.

 

Does Brazil have financial thresholds in terms of determining which kind of modalities you’re going to use, if you’re going to use the auctions or you’re going to use the restrictive competition ones?

Yes.

 

Or is it just done by case-by-case scenario?

No. There are thresholds and one of the things that the auditors look at and that we also, that is part of some of our corruption measures is in fact whether they basically fraction the purchase to avoid those thresholds. So there are thresholds.

 

Because that is fascinating, especially, or even in Europe because the financial thresholds in Europe are quite high so you only apply rules for contracts above 100 and something, for goods and services 100 and something thousand Euros if I’m not wrong or more, or even more if you’re a local authority. And I’ve always thought that it was not a very good idea, especially on corruption grounds but I’ve been writing about that area over the last few years and I’m going to revisit it again and you’ve just given me a new piece of the puzzle for me to work with which is actually where there is more discretion for public procurers we are creating the conditions to generate more corruption or at least make corruption more likely?

So that’s what our findings suggest for the first time that at least in Brazil in this particular context this discretion tends to be abused, okay. Now how externally valid is that? As you say I think that’s…

 

Yeah, we don’t know?

That’s research, we don’t know, that’s right.

 

Okay. But in that sense I’m writing about these ideas and obviously this area of public procurement from a legal perspective so I can afford the limited availability of data to try and shape my thoughts, although obviously I want it to be sustained by evidence and…?

Mm, and of course, of course.

 

Right, okay. So I’ve got a few extra comments for you.

Sorry Pedro, can I, because you asked and I’ve been going off topic I think so the paper is about procurement but if you don’t mind?

 

Yes, sure.

One of the things we also look at is this idea that when you increase the audit risk service providers are more likely to actually show up for work, okay. So there’s, it’s not corruption in terms of stealing but corruption in the sense of you’re paid to do a job and you don’t do it so, and the reason we can do that is because the CGU audits they include that as a standard procedure for them to basically also try to assess whether for example in health service delivery the officials, and these are going to be doctors and nurses but also going to be health workers that are supposed to visit the health facility users preventively, so the idea is that it’s cheaper to prevent than to cure but to, the prevention requires that these guys go out there and visit the families to find out whether someone is ill so that the prevention can actually take place and that’s costly of course for these agents. So what we looked at there or what the auditors looked at is they ask a random sample of potential service users in a given area they make household visits and they ask “are you receiving these visits from these preventative healthcare workers?” They also ask “when you go to the health post is it open when it should be open and when you go there is the doctor there?” These kind of questions so, and they ask some questions in control and treatment municipalities, okay. Now what we find on that front is that although in the control group there is not perfect performance in the sense that people do report that the health post is not always open as it should, the doctor is not always there, the increase in audit risk has no effect on that performance, on that compliance with your stipulated job requirements. In stark contrast to what we find in procurement, we find this reduction.

So what we, we were puzzled by this, it was we didn’t expect that but then what we do in the paper is we obtained a standard economic model and we see whether we can rationalise these findings. And one explanation for these results could be that when you get audited in procurement basically you can’t argue with those findings, you know. If the auditor finds that the firm that you awarded the contract to doesn’t exist there’s not much more you can say, whereas if the auditor goes in and he finds that people report that the doctors were not in the health centre when they should have been, the doctors are going to say “no, I was there” and the audit finding doesn’t generate that kind of hard evidence that would make people work. So we think that ex post we can rationalise our findings so the fact that health service providers might just not be worried by the audit because they can always say “well no, no, I was there” or “I was in another room” or “I was, I don’t know, I was visiting a family out in the field” so it just doesn’t generate the same kind of evidence. And so that, you know, what that, it just, I guess it also, it’s not, obviously it’s not clear how general the finding is but it raises this issue that the quality of audits can be useful to monitor certain things but not others. And then that has implications for audit design so perhaps the auditors can save these visits because at the end what’s being generated is not really useful that much.

Anyway, so I just wanted to mention that. I know this is a procurement audience but because the increased audit risk happened for everybody in the entire municipality and because the auditors look at these things as a matter of routine we thought that it’s useful to look at that as well, even though it’s a null result we think it’s interesting.

 

That is interesting and I was wondering as I was listening to you if it may be that the consequences are actually as far as I know very different in case you are found involved in corruption, actual corruption in a public procurement process or just not doing your work properly?

You’re exactly right. So there’s an additional reason which is that if you’re found stealing from any public procurement you can basically go to jail, whereas even if you’re convicted that you were shirking on the job you might lose your job at most.

 

At most, yeah.

You know, the sanctions are just less as well. So you’re exactly right, that will be another explanation. To be fair there’s also the explanation that as you said earlier, right, information might not have travelled to those guys, could be that the…

 

They did not know?

They didn’t know they were not aware of this audit thing and for them they were not treated in some sense. And, you know, we won’t know [laughter] so…

 

I’ve got another question here, a little bit more specific is that I think it’s on page 11 or 12 that you say that corruption or at least stealing less today does not imply that more can be stolen tomorrow, so you argue that because the federal funds cannot be saved for periods longer than the budget year that there’s no possibility that someone will not steal today so that they can steal more the next year because they’re just going to lose the budget?

Uh huh.

 

I agree with you, I think that is true but I was wondering if you found something within, any difference within the same budget year? Let me explain, what I’ve seen working with public procurement officials in different countries is that sometimes when we get close to the end of the budget year you still have some of your budget left and you need to spend it, you need to spend it and you don’t usually spend it in the best way possible, so that would increase the likelihood that someone would be tempted to take some money or be corrupted in one way or another. So what I’m wondering is if within the same budget year or fiscal year have you seen any differences or potential differences in terms of one is more, when is corruption more likely to occur?

Mm. It’s an excellent question but without data…

 

Without data, yes.

We are bit limited on that because we don’t know basically when the purchase was decided, you would need monthly data in some sense, right? You need to know when was, and that we just don’t have. I think for other countries, I think for Chile there’s actually a study that basically shows what you’re saying. Maybe you had that study in mind? But so …

 

No I don’t, I don’t. I would like to know more about it so if you know it just give me the information afterwards and I’ll look it up?

Yes. I don’t have the authors off the top of my head right now but this has been empirically documented. Now I think what they look at if I remember correctly is just, do they, so I’m not sure they actually have audit findings on these transactions, I’m not sure if they can say that it’s more corrupt if it’s spent towards the end of the year but things do happen that are special at the end of the year, that much I remember.

 

Okay, very well. One final question, next steps and future research plans in this area, do you have any? Do you think that you’re going to still revise the paper one more time or…?

Well the truth is that it’s not published yet so I’m sure I’ll revise it, that’s just, but that’s nature of the game and, you know, in fairness to the referees it usually gets better. But the paper is what it is. What I do find though is that academically people tend to, at least the referees I’ve got, they often say “well it’s not really surprising, we already know that an increase in audit risk will reduce corruption, it’s sort of obvious” and I think that’s just wrong and I think it’s actually, it hurts public policy because you just assume these things and you don’t study it. The fact is that we don’t, we know very little about the incidence of corruption in procurement and service delivery more generally across the world and then what works to reduce it, if it has worked in one context is it going to work in another one? We just have no idea. So I, there’s the, the short answer to this is that I think I’ll continue working on this topic trying to see whether more monitoring or other kind of monitoring or even also incentive schemes can improve service delivery and reduce corruption but I’ll have to find willing partners and that’s, you know, that is the, you have to work, to do this kind of stuff you need connections with governments and that’s, that is sort of the major constraint, okay. So I can say I want to do that, the question is going to be am I going to find governments that are willing to do that kind of stuff and I can just, all I can do is try. So that’s one.

Another area where this can go where it’s a little bit, a different question is the fact that public procurement has in recent years moved very much online so e-procurement where from a monitoring perspective what’s interesting about that is that there is much more information now about let’s say the awarding stage, right, so you can basically go online and see what kind of contracts are available for which you can bid, you can see afterwards, you know, who won, so there’s just much more transparency in the entire awarding stage and I have a project with two other co-authors where we try to see when, under this increased transparency in the awarding stage corruption or irregularities get shifted more towards the execution stage because in order to know whether the price agreed was actually the price paid or whether the quality was according to what was agreed or not, whether it was substandard quality in the execution, you actually, you still need an audit. This whole movement of e-procurement hasn’t changed the fact that we know very little about what’s going on in the execution stage and there’s our worry that just the focus of corruption might shift towards execution. So that’s just, you know, putting sort of a question out there that we’re working on but we haven’t, I don’t have any results but I think that’s, in the future that is interesting, So less or increased transparency at the awarding stage might actually shift, just shift things around and overall towards the execution and that, and overall you might not get much of a benefit. So that’s a worry that we have and we want to investigate that.

 

That’s a very good couple of points that you’ve made there for us to conclude the show on. I would just like to say that I agree with you that the more transparency that we have during the awarding stage, the more likely it is that whatever, let’s call it irregular behaviour or illegal behaviour taken by agents is going to move elsewhere. You are right in looking at the execution stage because execution stage unfortunately so far has not been subject to as much scrutiny as the award stage and that for me is a mistake. It’s a big, big mistake in terms of policy and I’ve been against that view of procurement, that procurement is only the award stage and not execution for many years. On the other hand something that you may not be aware of is that at least in Europe under the new EU Directives it’s now possible in variable terms unfortunately in my view for contracting authorities to have conversation with suppliers even before they open the award stage, so that means that there’s another, there’s another moment where corruption can occur which is actually before the award procedure. So even if you have let’s say an open procedure, so what you guys would call an open auction for example, if you have an open procedure, very transparent, very clear, you look at it, just at the procedure and you will not find any corruption but you don’t know that for example one of the bidders was told in advance what was going to be included there and for example shaped the actual award criteria.

Right, so the buyer can tailor the call to whatever they, and you’re saying this is legal in Europe?

 

What I’m saying is that it’s legal nowadays for the contracting authority to have conversations with suppliers before it launches a tender and the way that it is drafted in the directive it’s very, very liberal and way too much liberal for my taste.

Interesting. But so is this a change that’s happening?

 

Yes.

Or are you, ah.

 

Yes, it is a change. I mean economic operators know but contracting authorities have been asking for this for many years, they say that if they are forced to use an open procedure, say an open auction, that is very difficult for them to know in advance what they really need and how to spec it out correctly so they need to talk with the market in advance, so if they talk with the market in advance the procurement is going to be better and I just say bollocks, you’re just creating the conditions for undue influence to occur. But that’s another area that is coming up at least as a potential area of research in corruption in public procurement.

Well very interesting. Thanks a lot for letting me know. Well, you know, let’s keep in touch. I mean clearly I have a lot to learn about what’s going on in Europe.

 

Okay. Thank you very much Stephan, was a pleasure talking with you.

Thanks Pedro. See you.

#16 - Guilherme Lichand (Harvard University)

Can a  little bit of corruption be good for your health?


iTunes

Guilherme Lichand's Harvard page and his job market paper.

 

Transcript

It’s great to have you here Guilherme. Your paper it’s fascinating and thought provoking. Why did you decide to look into the issue of corruption in the health sector and its potential downstream effects?

Great. The idea – not only in health, but more broadly – is that while we have increasing evidence of what tools can most effectively fight corruption (in particular, audits have been shown to significantly decrease missing government expenditures), we have much less credible evidence about what is the effect of those tools on the outcomes that corruption is supposed to detrimentally affect. The distinction matters, because bureaucratic performance has multiple dimensions; rent capture is just one of them. On the other dimensions we have, for instance, the quantity and the quality of public goods’ provision. So while, on the one hand, limiting the extent to which bureaucrats can capture rents should mechanically increase resources flowing towards public goods’ provision, on the other hand it might decrease incentives for bureaucrats to exert effort on those other dimensions. Theory predicts that the implications of deterring corruption on public service delivery should be ambiguous, and, hence, we try to understand whether deterring corruption helps or hurts public service delivery. That’s the prime motivation on the paper.

 

Any reason why you picked Brazil over any other country that you could have done this research on?

 

Brazil introduced an anti-corruption program nationally in 2003, so it provided a great opportunity to try to answer this question. First, because the programme was introduced right at the mid-point of the Mayor’s political term. The term is from 2001 through 2004, and the programme was announced exactly in January 2003. Second, because these audits are retrospective. Basically, when auditors go there, they follow the paper trail of every investigation, at least up to three years prior to the time of the audit. This way we have the data we needed to try to answer this question. That also takes us to the point of why health. The answer is, first – given the audit data that we needed to code – because we needed to restrict that attention to a particular area of public policies, because it is a huge amount of data. Second, because of outcomes – availability is very good at the municipal level for health. We have a lot of indicators at the municipal level that are measured for all municipalities every year, and that was really what we needed in terms of data requirements to try to answer this question.

 

What's the main finding of your paper?

On the one hand, we find that the program greatly reduced corruption – and by this I mean procurement manipulation, off-the-record settlements with vendors, and resource diversion. These kind of problems, according to the audit reports, are greatly decreased by the introduction of the program. But on the other hand, what happened is that the program also dramatically deteriorated the quality of health indicators – and by this I mean hospital beds per thousand inhabitants, immunisation coverage, or household access to piped water and proper sanitation, in comparison to other health indicators that involve much less procurement. So to anticipate the results, what basically happened is that, it seems, we thrown the baby out with the dirty bath water. The programme was a major institutional improvement in terms of transparency, but by focusing so much on corruption – both in terms of public opinion and administrative punishments –, what seems to have happened is that spending fell by so much that corruption per dollar spent has actually increased after the program.

 

Really?

Yes.

 

So let’s break that down into sub-questions or comments. So the audit programme did reduce corruption so you guys were able to measure that, right, okay, so that is one element. So in terms of the let’s say, first degree objective of the programme it worked as it was supposed to work in terms of reducing corruption at least in absolute levels if not, as you were saying, in relative levels per spend.

So when we look at the percentage of investigations that were coded as corruption, looking at these audit reports, indeed it is the case that the baseline incidence of corruption is decreased by half. It is a major effect. Just to give you a sense, among those transfers that involved a lot of procurement – we look at how programs earmarked to specific spending, like buying hospital beds or contracting hospital reforms, those sort of transfers –, before the program, about 37% of our investigations were coded as corruption, some sort of the problems that I previously mentioned. After the program, that rate falls to about 12%, and this is the same rate for those audit transfers that did not involve procurement at all – like those earmarked to pay wages of doctors or local procurement staff. So, basically, the idea is that, as you said, if you just focus on corruption, then the program would do exactly what it was supposed to do: every investigation now had much less of this procurement problems that were very prevalent beforehand.

 

But on the other hand?

Yes, on the other hand... before we even tried to look at the mechanism, if we look at the outcomes – because, you know, corruption fell by so much you might think that health indicators would become much, much better after this –, what we find is that these health indicators (comparing again these two sets of transfers: those with a lot procurement and those basically with no procurement) become significantly worse. Just to give you a sense: in a cross-sectional comparison, the magnitude of the effect is equivalent to losing between half and all of the support of the federal government towards municipalities’ health budget. Because municipalities depend so essentially on federal redistribution to be able to provide those local public goods, this effect might mean that they have no budget left, and this is consistent to the effects when we look at spending. After we found these effects on the health indicators, we wanted to understand the mechanism. What was happening? Why was corruption going down but health indicators becoming worse? So we looked at two other dimensions of bureaucratic performance. First, quantity – proxied by spending. What we find is that using the same strategy, differential spending across the two sets of transfers fell by at least 50% after introduction on the program. This effect is consistent with the deterioration of the health indicators. And just by residual, since corruption fell by about 30 percentage points (the percentage of investigation coded as corruption), and since spending fell by at least 50%, this must mean that corruption per dollar spent must have increased by at least 20 percentage points. So that’s where this figure comes from.

We also look at quality: from these audit reports, we can also look at implementation problems. We find that, basically, infrastructure problems abound, infrastructure is no longer functional (or high quality), and also medication stock problems abound after the program. Because you’re not spending, then all these problems are coming about and this is consistent with the overall story that (i) either because bureaucrats can no longer capture rents they have less incentive to exert effort in these other dimensions or, (ii) and this is something we might talk more about, these procurement guidelines are so complex and the quality of these local procurement staff is so low that it’s now easier just not to spend and blame the federal government for not transferring resources than perhaps running into a procurement mistake and being labelled as corrupt.

 

We’ll address that in a minute. There was something that I was listening to you and I think it’s important to probe a little bit further there as well, which is it makes sense that the outputs or at least a proxy for the outputs, would go down if spend goes down as your analysis show so, you know, fewer beds per a thousand inhabitants, worse outcomes in terms of immunisation or the availability of medicines, all that makes sense because you’re talking about the health budget and how public procurement is really in the health system work. What I didn’t understand was the relationship with the access to piped water and as well to the waste treatment so how do you connect those two elements? Because if I get it correctly, is the audit then to the whole municipality or just to the Health Department?

So these audits look at overall expenses, it’s just that within health and education they look at the whole set of transfers whereas in other areas they look at a sample of the transfers. But the thing is, there are some health programmes (in Brazil at least) that are targeted to improving the connection to piped water, the quality of water and sanitation, and so when we choose indicators we look at what indicators the Health Ministry uses for monitoring and evaluation.

 

No, that makes sense now, it’s just that I wasn’t seeing the connection right immediately because the other ones are obvious and that one I knew that there must be a reason there, I couldn’t just find it. In terms of your analysis, you’ve done something that I find very interesting, is that you decide to separate the analysis in accordance with different procurement intensity so you’ve looked both at procurement intensive processes and procurements not intensive processes if it makes sense, why did you do that?

So you know the program involves random audits. So one might think that you could just look at the effect of the audits themselves on corruption and on these health outcomes to understand whether the mechanism we propose makes sense. Now, one problem with this is that, although we can indeed do exactly what I just said, most of the effects of procurement might perhaps have happened at the announcement of the program, even before any audits actually took place. So to understand the overall effect of the program we have to resort to a differences in differences strategy. We can compare (because we have the retrospective data), municipalities before and after both the Mayor and the local procurement staff learned they could be audited – so that’s one dimension of comparison. But if we just did these over time comparisons, many other things changed in 2003, so this is not enough to really pin down the effect of the program. So the idea is to look within municipalities, to different sets of transfers: those that involve a lot of procurement, and those that do not. The idea is that, first, the program should have had a differential effect across these transfers. Because most of corruption is linked to procurement problems, it should have had a higher effect on those transfers that involve a lot of procurement. And second, any differential in corruption across these two sets of transfers should have remained constant otherwise. That’s the assumption for relying on this strategy to try to pin down the effects of the program. We also use recent nearby audits as a complementary strategy (because they are random) for trying to assess the same mechanism. Roughly, the effects that we find are consistent. So both the overall effect of the program and those of audits themselves are in the direction of decreasing corruption, but also of decreasing spending by so much the health indictors actually become worse.

 

Yes, okay. Speaking about the audits, are they really random, i.e., is really a lottery or is this that the official claim from the system for the outside world is internally they pick who they want to do the audit to?

That’s a good question. This program is a joint venture with the National Lottery, so the draws are in television. Every two months or so (at least in principle), about sixty municipalities were supposed to be drawn. The first papers to exploit variation from this programme were by Claudio Ferraz (PUC-Rio) and Frederico Finan (Stanford). They try to understand if these audits are indeed random, so they look at a broad range of variables, political parties included, and they don’t find significant differences. Each municipality has a 1% probability of being drawn, they do it in this way every draw. What we had to do involved an extra step: to show that not only audits are random, but also that conditionally on municipality’s characteristics, recent audits that happened nearby are also random, so that we can exploit not only if a municipality itself is audited, but also if a neighbour is audited, it this raises the perceived probability of you being audited in the future – out of a behavioural mechanism – to look at the effects of interest.

 

Yes, the reason I was asking this is because in Portugal theoretically you are selected randomly to provide evidence about your tax situation and it was claimed originally to be a lottery but I mean the same people were getting the letters from the IRS, the Portuguese IRS, every year so it was very clear quite quickly that there was no lottery involved. What I did not know is that in Brazil actually it’s really a lottery and it’s done at least apparently above board.

 

Right. So there are many oversight programs in place, some of which are not based on random audits. The state tribunals do a bunch of audits which are motivated by either some event or even by a private investigation, so it’s an overlay of different institutions. This program specifically is based on random audits, and that’s what makes it nice at least for the investigation in this paper.

 

Yes, it makes perfect sense because if it’s random it’s one less variable or one less bias slash problem you may have with the data that could be influencing you.

Exactly.

 

Very well, moving onto another topic which is how transferable is this finding or these findings to other areas of public sector that are also subject to procurement analysis of expenditure, particularly areas that are very procurement intensive like say, for example, public works which is always usually the one that comes to mind? 

Our thought is that there’s nothing really specific, it seems, about this program in Brazil that would prevent the findings from being extended elsewhere. First, this kind of bureaucratic politics through which the federal government transfers resources that would fund the provision of local public goods, with a local bureaucracy which is responsible for running procurement before these resources can reach their final users – citizens – in the form of these public services, is the modal workings of public bureaucracy in the developing world, it’s seems to us. And there is evidence from other settings that are very consistent with this findings. There is a paper, from 2015, in the American Economic Review, for instance, about a program in Nigeria, showing that when it monitored bureaucrats on several dimensions of bureaucratic performance then they exert less effort in those dimensions; they try to divert efforts to dimensions that are less subject to monitoring. This is consistent with this idea that whenever you have this layer – bureaucrats that must procure resources before they can actually reach citizens –, you are subject to this kind of tension between actually procuring / spending, on the one hand, and on the other hand, making proper use of public funds. So the only difference that I can foresee is when, in some settings, resources are directly transferred to the end users. There is a paper, from 2004, with data from Uganda, in which they tried to investigate the effects of a newspaper campaign aimed at ensuring that the federal resources would reach schools. They claimed that, in Uganda, before the campaign, basically the modal transfer to schools was zero. So most schools were getting basically nothing out of what they should actually get. After the newspaper campaign, then citizens put pressure on the federal government, these resources are really going to the schools and then educational outcomes improve. But there you see the difference: there is this layer of bureaucrats having to procure which is not present, so I think that explains the difference, in my opinion. In any case, wherever you have bureaucrats, the same logic should apply.

 

It’s very interesting for two reasons, the first one is that effectively its principal agent relationship issue that is at hand and one of the things that I have noticed that is mentioned on your paper is that because taxes are collected centrally but spent locally that there is less emphasis or less say pressure from the actual supposed beneficiaries and taxpayers in that local area to actually make sure that the money is well spent, which is the difference with Uganda, it’s a big difference.  

That’s right. I mean, in Uganda, perhaps, in this particular sense that it’s not raised locally so that citizens have a lesser incentive to pay attention to it, I think this part is exactly the same. It is just that when you put it on the newspaper, then citizens have a higher incentive to monitor. There, this makes resources get to the schools; and that is where they should get at the end, so you’ve already done the job. In Brazil, there are other papers that show that when you put these evidence of corruption in the newspapers, citizens punish politicians. There is a paper showing that the probability of re-election falls by about 30% after – comparing two municipalities that are equally corrupt, but one was exposed just by randomness before the election, and the other after it. So citizens do take this information into account in both cases. It’s just that, in the Brazilian case, this is not enough. In Brazil, the only way a bureaucrat can lose their job and go to jail or have to pay a fine out of their own pocket is through this program. So you can imagine how high the stakes are for making a mistake or actually capturing rents whereas the upside of doing your job right and making sure the resources are properly spent, currently, is very low. The rewards from doing a good job as local procurement staff are very low, there are no incentives for public service delivery but there are very high punishments for corruption now.

 

That’s very interesting and it makes perfect sense. Another point I picked up  was that you were talking about, from a developing world perspective, and saying “oh this is how we organise let’s say the health sector or the way that procurement is done in the development sector”. Having said that, I mean the same thing happens in developing nations, I mean, for example, here in the UK, thinking about the health sector in England you have a number of different authorities, Health Authorities, which are called Health Boards, each one responsible for one, two, three hospitals. In Wales you can have the same structure with different people. In Scotland as well, in Northern Ireland the same as well in an even smaller number and in the countries that I know, Portugal, Spain and to a certain extent France and Germany, there is also a big effort to try and put the decision, the decision making even in procurement as low down the chain as possible and as close to the citizens as possible as well. So, for example, in Spain you’ve got regions as you have the states in Brazil, and the regions have or are responsible for their own health sectors. So I think that if your findings are indeed transferable they can be transferable not only in a developing world setting but also on a developed world setting as well.

I don’t have much to say about this; what I can say is that what we find is consistent with an old discussion from the 1960s. For instance, Sam Huntington said that during the process of development of the US and the UK, corruption was one of the main drivers of development. Professor Steve Kelman, from the Harvard Kennedy School, very often in the media emphasises that perhaps we might be over-monitoring – paying too much attention to complying with too rigid procurement guidelines, versus you know, paying attention more broadly to the quality of public services. So, you know, from the paper, there is a limit to what we can learn about other settings, but I think it is consistent with the more general discussion here.

 

Yes, picking on that and moving to the next question, one of the things that came to mind as I was reading your paper is that the fact that the bureaucrats, the civil servants, know that they are being monitored or can be monitored and can be audited and hence the change of practice and corruption goes on but also the delivery goes down as well with it, it maybe that even if those agents are not corrupt by themselves the fact that they know they are subject to more scrutiny and more accountability actually raises the stakes for them so if they can avoid doing a procurement process that is deemed risky or could be risky, as you said, it’s a question about incentives, if they don’t have the incentive to provide a good service they might as well avoid the risk of getting the fallout from anything going wrong even if it’s not, you know, directly corruption.

 

I think you may be perfectly right. We cannot fully attribute the results to the fact that local procurement staff can no longer capture rents to the same extent, such that there’s just less incentives to do everything else. In our paper, we are not able to separate between the two stories. Indeed, from speaking to both auditors and local bureaucrats, these procurement guidelines are so complex and, as I said, the quality of the local procurement staff is so low, in comparison, that it’s just easier not to spend and put the blame somewhere else for the lack of delivery than to run this risk of being mislabelled as corrupt from running into a procurement mistake. So the general message is that not only perhaps we should refine how to balance these incentives, as I’ve said before, but also try to raise the capacity of these local procurement staff. With any capacity building program, we shall see have positive effects on both spending and corruption, if it decreases the likelihood of these procurement mistakes. And also, perhaps, we could redesign the program to focus on larger transfers, because if you think about the rational decision to capture rents, the incentives are really there regardless of the probability of being caught for the really large transfers. And these are the ones for which the scare effect on spending from the risk of being audited is not going to matter much. Basically, if you think what the program is doing... if there is this component of being afraid of running into a procurement mistake that is scarring bureaucrats out of very small transfers – for which they probably wouldn’t have been corrupt anyway and, even if they were, the consequence of that for the public budget would be very small. So there is a question there of the actual design of the programme and also of capacity building at local level that I think these findings raise.

 

Yes, I think those issues are raised and raised a lot at the end of the paper and it’s certainly something that echoes what has been argued here in Europe and you think that in Europe procurement is very good and full of, people full of capacity, that is not the case. Even here in the UK there’s a huge distribution in terms of the quality of the contracting authorities and also the quality of the people that actually deal with procurement on a day-to-day basis and in fact here in the UK with the budget cuts over the last five years or so,  procurement capacity has probably gone down in most places because they’ve lost staff, senior staff, and they haven’t been able to replace that senior staff. So it’s very interesting because when people ask me “Where should they save money?” I say, “Well you should save money in procurement certainly”, but probably the best way in the long run to do it is to actually up-skill your staff to make sure your staff is competent and can deal with complex procurement guidelines which are always in Europe as probably as complex as the ones in Brazil. So it’s not only a problem in Brazil I would say.

Right, it makes sense. It always comes to mind to me, it’s not only about having higher-skilled people in your staff, but also about having models for basic projects and everything. For instance, one phenomenon we see there, just anecdotally (and I’d like to document this), is that when a capital city procures a complex public work, like a school or a big piece of urban infrastructure, a lot of the other cities copy, procure the same.

 

Really? 

Because they don’t really have the skill to write the terms of reference. This is a very complex thing. So this just comes to show that you could do more. Of course it comes out of not enough skill from these local procurement staff in these other municipalities to do the same on their own. But, also, if either the state level or the federal level provided these basic projects as a public good, then they could use as a basis for what they actually did. That’s also an easier way, and you have economies of scale in doing this. Of course, at the local level you cannot do it but that would be a huge contribution that the federal level, for instance, might do besides just trying to do small training programmes at the municipality level.

 

Well, speaking anecdotally as well here in Europe the same thing happens which is I talk with procurement officers very regularly and they will say to me, “Oh yes, we’ve been using this, let’s say, this template that we got from our colleagues in a Council down the road”, and I just say, “Okay, so what quality control have you done on that?” “They’ve used it so I suppose it’s good”, yeah, right, okay, so.

So imagine if the federal government provided this template? That could be a huge contribution.

 

Yes, I agree with you, it’s a good idea, I never thought about that in that sense, I mean there’s been some attempts to try to do stuff like that or a little bit like that but very much on an anatomised basis and not on a very centralised and very controlled basis which I think would be necessary. Another thing that I think it’s relevant for the discussion, some colleagues of mine who are in Wales at my previous employer, Bangor University, they looked into the procurement practice of three small Local Authorities, so three small municipalities here in Wales and they found 600 people with buying power so with the capacity to buy stuff and obviously they were subject to rules, some of them are internal to each municipality, some of them are national and some of them come from you know from the, what we call the premium directives but I just ask myself how will three small Local Authorities be able to up-skill all these people to make sure that they are good enough, they just, even if bringing into the level that they’re just average procurers. Or is it possible, it’s pretty much impossible and the only way I can see it, is that if you actually reduce the number of people that are involved in these processes and you increase to a certain extent the transparency and the monitoring to make sure that you are actually capturing the information.  

Right, that almost comes down to the organisational aspect of it.

 

Yes.  

It might involve a lot of people, because these municipality in Brazil might be very complex, with up to 20 different secretariats. And that makes sense, because they’re responsible for so many local services. But it’s true that it’s impossible that you would think that each of these secretariats are going to be equipped with the best procurement staff, so you should expect that perhaps the secretariat of planning might centralise all the best staff in procurement and everyone has to run procurement through them. But it comes down to systems, and retention, etc. And, you know, the discussion is usually not there.

 

I agree, I agree, I fully agree with you. Well our half an hour is over so I’d like to thank you for this very enjoyable conversation we had, it was great. As I said you’re the first one from economics that we’ve had here in the podcast and I hope you’re not the last one before we finish the series.

Great, my pleasure.

 

Thank you very much for coming Guilherme.

Thank you.