On Monday morning, the public works department reviews its dashboard. The northern neighborhood logged 40 pothole reports this week. The eastern neighborhood, four. The decision looks obvious: the crew goes north.
What the dashboard does not say is that the 40 reports in the north correspond to six potholes, reported again and again by the same residents, on streets that were repaved three years ago. In the east, the four reports come from streets with almost no pavement left and an open storm drain in front of an elementary school. Nobody else reported because people in that neighborhood work far away, have no time to call, and no longer expect anyone to come.
The crew will do good work in the north. And the municipality will have allocated its scarcest resource using a criterion nobody chose: who complains the most.
Report volume measures two things mixed together: how much of a problem there is and how willing people are to report it. If the municipality does not separate one from the other, its work queue ends up ordered by the second.
The intuition that not every neighborhood reports the same way has empirical support. In 2021, Constantine Kontokosta and Boyeong Hong of New York University published a study of Kansas City, Missouri, in the journal Sustainable Cities and Society. They cross-referenced the city's 311 reports with a resident satisfaction survey, code enforcement records and visual assessments of the condition of each street.
With that data they estimated how many street reports each area should receive given the actual state of its pavement, and compared that with the reports that actually arrived. The results are hard to ignore:
- Areas that under-report have the worst streets. The average pavement score was 68.27 in over-reporting areas and 48.00 in under-reporting ones. According to the authors, that 20-point gap is equivalent to two years of deterioration under the city's maintenance standards.
- It is not that they are satisfied. Satisfaction with street maintenance was lower in under-reporting areas (2.15 versus 2.91). Need is higher both in the technical measurement and in residents' own opinion.
- The gap in reports is an order of magnitude. In 2017, under-reporting areas logged 0.0056 pothole reports per street mile per parcel, versus 0.058 in over-reporting ones.
- Duplicates inflate the signal. In over-reporting areas, the same pothole received more than one report about 19% of the time. In under-reporting areas, less than 4%.
The most useful finding for a municipality is about the type of problem. Lower-income areas do not stop reporting; they report something else. They use the channel for what is most serious, such as dangerous properties, and set aside what they perceive as a minor nuisance, even when their streets are worse. The authors conclude that the representativeness of 311 depends not only on how much each area reports, but on the severity of the problems it experiences and chooses to report.
A second study reaches a similar conclusion with a different method. Zhi Liu and Nikhil Garg of Cornell Tech, together with Uma Bhandaram of the New York City Department of Parks and Recreation, analyzed more than 100,000 reports to that department and more than 900,000 to Chicago's transportation and water departments. Published in Nature Computational Science in late 2023, the work found that, even when comparing incidents with similar characteristics, some neighborhoods reported three times faster than others. In New York, reporting speed rose with population density, education and income.
Kontokosta and Hong tested three ways to distribute pothole repairs across Kansas City: using only the number of reports, using only the visual pavement assessment, and using an indicator that adjusts reports for each area's propensity to report.
With the first method, the one any dashboard that counts tickets uses, the lowest-income quartile receives fewer repairs than its street conditions would justify. With the adjusted indicator, most repairs go to the two lowest-income quartiles, which have the worst pavement. The visual assessment also corrects the distribution, but the authors warn that it is expensive, slow and lagging: by the time an inspection records the deterioration, the deterioration has already happened.
The median time from report to repair was the same in both groups: four days. A municipality that only measures response times would conclude that it serves everyone equally. The inequality is not in how quickly a report is handled, but in which problems ever become a report.
In Mexico, the gap between demand and capacity shows up in the surveys. According to INEGI's National Survey of Government Quality and Impact (ENCIG) 2023, only 27.2% of the urban population aged 18 and over said they were satisfied with the service of streets and avenues, the lowest level among the basic public services the survey measures. Just 16.6% said the streets in their city are in good condition, free of potholes and sunken or open storm drains, and only 13.1% said potholes and open storm drains are repaired immediately.
When demand far exceeds crew capacity, the criterion used to choose what to handle first matters more than any improvement in speed. If that criterion is volume, the municipality is not deciding; it is letting those with more time, more access or more persistence decide.
Bias is not introduced by residents, but by the way the municipality turns reports into work:
The channel. If reporting requires downloading an app, creating an account or calling during office hours, those who can do so are the ones who report. Every barrier to entry filters out the people who work the longest hours away from home or have less digital access.
Duplicates. Ten reports of the same pothole are not ten problems. If the system counts them separately, the area that insists appears to have more demand than it does, and the crew may end up sent to the same spot several times.
Categories without severity. If a pothole on a dead-end street weighs the same as an open storm drain in front of a school, the only way to tell them apart is volume. Severity has to be a property of the report type, not an operator's impression.
The direct call. The report that comes through a council member, a neighborhood leader or the director's phone skips the queue. Sometimes for good reason. But if it is not recorded with the same severity and the same due date as the rest, nobody can know how much work is assigned because of pressure and how much because of need.
No municipality can inspect every street every month, but it can change the rules it uses to order the work.
Assign severity to every report type. An open storm drain, a drinking water leak and a pothole on a main road do not weigh the same as a flickering streetlight on a pedestrian street. Severity should be defined in the catalog, together with the departments, and inherited by every ticket.
Work with due dates, not with age or volume. Every ticket should have a deadline that depends on its type and severity. The day's queue is ordered by what is about to expire and what is most serious, not by who called the most times.
Link duplicates instead of counting them. A new report near an open one of the same type should be added to the existing case. The number of affected residents is useful data; the number of repeated reports is not.
Lower the barriers to entry. Reporting should be possible through the channels people already use, with no account and no new app, and by phone for those who do not write. The more even the access, the less biased the volume.
Read the map in two layers. A heat map by volume shows where people report. One by severity and by tickets at risk shows where the problem is. Areas with few reports but high severity are the first candidates for verification rounds.
Record exceptions. If a case is handled out of turn, the ticket should record who asked for it and why. That way the leadership can review each month how much of its work responded to the queue and how much to pressure.
Compare with data that does not depend on reports. Pavement inventories, supervision rounds or case reopenings help detect areas that under-report. We have already explained why a patched pothole is not a repaired street: recurrence is another signal of where the real need is.
Agora, Intello's citizen services platform, is designed so that priority is set by the problem, not by persistence.
The concept catalog defines, for every report type, its severity, its estimated time and the departments that handle it. When a ticket is opened, via WhatsApp or from the panel, it inherits that severity and its due date, and the first department becomes the owner. In the call inbox, emergencies and possible crises appear first.
To keep volume from inflating, the WhatsApp assistant confirms what happened, where and with a photo, and if there is already an open report nearby, it links to it instead of creating another one. To make reporting even, Agora receives reports via WhatsApp, Messenger, Instagram, web chat and phone, without residents downloading an app or creating an account, and transcribes calls to propose the ticket.
In tracking, every ticket shows an attention status light: on time, about to expire, expired or met, and the log records who changed what and when. Analytics brings together tickets about to expire and expired, a heat map by volume and severity, a map of tickets at risk, and resolution times, reopenings and backlog by department. That is what lets a director handle first what is about to expire and prioritize by due dates, not by whoever calls the loudest.
- Does every report type in your catalog have a defined severity and due date?
- Does your dashboard count reports or unique problems?
- Do you know which neighborhoods have few reports and streets in poor condition?
- Can people report without an app, without an account and outside office hours?
- Is it recorded when a case is handled out of turn because of a direct request?
- Do you check whether response times and the number of repairs are even across areas with similar conditions?
If several answers are no, your work queue is probably ordered by volume, not by need.
Citizen reports are a municipality's best source for knowing what is happening on its streets. But they are a sample, not a census, and the sample is not even. Treating volume as need turns a difference in the willingness to report into a difference in the quality of service. Prioritizing by severity does not require giving up on reports, but putting rules on them.
If your municipality wants to order its citizen services by the severity of the problem and not by who complains the most, see how Agora works, review how Tribuna brings the same traceability to public safety and municipal justice, or request a demo.
- Kontokosta, C. E. and Hong, B., Bias in smart city governance: How socio-spatial disparities in 311 complaint behavior impact the fairness of data-driven decisions, Sustainable Cities and Society, vol. 64 (2021)
- Cornell Chronicle, Method may improve cities' responses to resident service calls (December 2023), on Liu, Bhandaram and Garg in Nature Computational Science
- Liu, Z., Bhandaram, U. and Garg, N., Quantifying Spatial Under-reporting Disparities in Resident Crowdsourcing (arXiv:2204.08620)
- INEGI, Encuesta Nacional de Calidad e Impacto Gubernamental (ENCIG) 2023: principales resultados