[HN Gopher] What I learned in two years of moving government for...
       ___________________________________________________________________
        
       What I learned in two years of moving government forms online
       (2018)
        
       Author : williamsmj
       Score  : 220 points
       Date   : 2021-03-18 04:52 UTC (1 days ago)
        
 (HTM) web link (medium.com)
 (TXT) w3m dump (medium.com)
        
       | effnorwood wrote:
       | Summary - "We are fucked"
        
       | vorpalhex wrote:
       | I was in tears with the amount of manual labor on easily scripted
       | tasks the author spent. Months of work that could have been
       | scripted in a few weeks.
       | 
       | You can't script the human element out, but you can save a whole
       | bunch of time clicking through pdfs looking for forms and filing
       | github issues.
        
         | beckingz wrote:
         | How would you go about programmatically determining if a PDF is
         | a form or not?
        
           | abraae wrote:
           | Our product automates pdf forms in a specific area (HR).
           | 
           | From experience, the presence of form fields in the PDF will
           | tell you that it is a form, but the converse is not always
           | true.
           | 
           | Plenty of people flatten their pdfs to remove the form
           | fields, or maybe they were never there in the first place.
           | 
           | I believe it's sometimes driven by misguided attempts to stop
           | people from "hacking" (aka automating) them.
           | 
           | Or pure incompetence.
           | 
           | Some other things I would have like to see in the article
           | would be some details of how they handled signatures. We do
           | electronic signatures but it is certainly an area where
           | accessibility comes into play.
        
           | prepend wrote:
           | Mechanical Turk?
           | 
           | HTTP get each pdf, look for tillable fields? Flag those for
           | review.
           | 
           | Even manually, 2 months to review 2000 links is a long time.
           | I don't think the author meant he literally sat there for 40
           | days, but likely did other stuff too.
        
           | hoten wrote:
           | Puppeteer should work. I believe PDF form controls are turned
           | in HTML form elements, which can be queried by Puppeteer.
        
         | williamsmj wrote:
         | I bet the author learned a huge amount of valuable context and
         | domain knowledge by reviewing so many forms "by hand".
        
       | oezi wrote:
       | "All of our submissions were handled via Google Drive." stopped
       | reading there
        
         | oezi wrote:
         | Clarification: How can anybody be serious about such an
         | approach that is clearly violating data protection
         | requirements?
        
       | hoten wrote:
       | Very interesting!
       | 
       | side note but... _why_ do so many articles like to repeat things
       | they just said in bigger text? It's usually somewhat spaced
       | apart, but in this instance the sentence is repeated (but BIGGER)
       | right after its written.
       | 
       | > On average, it took me about 30 minutes to make a digital form
       | and five weeks to meet with, earn the trust of, and get buy-in
       | from the employees who would use it.
       | 
       | > """ <same thing but BIGGER> """
       | 
       | I noticed this sentence is highlighted because it is popular; did
       | Medium decide to auto-bigify a pull-quote, or do authors decide
       | to do this?
        
       | idiotsecant wrote:
       | It sounds like you spent 16,000 hours to save residents 10,000
       | hours.
        
         | 3minus1 wrote:
         | This is funny, but each year residents should be saving more
         | and more.
        
         | jdefelice wrote:
         | "They also made government services much more accessible."
         | 
         | I would say the value of making the forms accessible was worth
         | it. Also with the flexibility to make changes to the forms in
         | the future will save even more time.
        
           | fatboy wrote:
           | Plus there's value in just making things better! Not
           | everything should be viewed through the eyes of capitalism.
        
       | xvilka wrote:
       | PDFs for forms are truly terrible and barely work[1] in anything
       | except Adobe Reader. Online forms are not without their problems,
       | especially from the maintenance point of view, but significantly
       | more accessible to the normal people, especially for the mobile
       | phones, if designed properly.
       | 
       | [1]
       | https://gitlab.freedesktop.org/poppler/poppler/-/issues?labe...
        
         | tomjen3 wrote:
         | I wished I had known that when I filled out a form in document
         | preview on my Mac a few hours ago, including adding a
         | signature.
         | 
         | The process was basically perfect and about as simple as can
         | be.
        
         | jimmaswell wrote:
         | I've filled out lots of PDF forms in Firefox and Chrome without
         | issue.
        
           | cratermoon wrote:
           | Because they license Adobe's engine to handle the PDFs
        
             | anoncake wrote:
             | Firefox uses PDF.js, Chrome's reader is based on Foxit.
        
       | specialist wrote:
       | Nice writeup.
       | 
       | One likely factor for Josh Gee's notable successes is his youth.
       | Because he's outside of the power structures and politics.
       | 
       | I started "adult" (office) work at 15yo. I got a lot done.
       | Everyone loved the geeky computer whiz who could fix stuff. I got
       | bonuses, perks, praise. This track record set me up for long-term
       | failure.
       | 
       | By the time I turned 25yo, I hit a wall. I was now an adult. With
       | my own power. So I was now perceived as a threat. Sadly, I was
       | completely oblivious to the changed dynamic. And I didn't have a
       | mentor or any one else to clue me in.
       | 
       | Since then, I've encouraged promising young talent to color
       | outside the lines (eg be bold, don't wait for permission) while
       | they still can.
       | 
       | Just from his writing style, I think Josh Gee will avoid my
       | mistakes.
        
       | ktsosno wrote:
       | The focus on not attempting to improve the forms or workflow
       | alongside the porting of the docs online is key. Cuts down on
       | potential errors, increases buy-in from stake holders and doesn't
       | turn this into some insurmountable task of being a consultant for
       | several dozen city departments.
        
       | ed25519FUUU wrote:
       | > _A submission emailed to them. A PDF of the submission dropped
       | into a Google Drive folder. A line added to a Google Sheet
       | spreadsheet with the submission data._
       | 
       | This all sound great for the city, but it makes me concerned
       | about privacy. At least with the form somebody had to be in
       | physical access to steal the information. Now they just need
       | access to one cog's Google drive.
        
         | cube00 wrote:
         | I also noticed the GitHub issues are full of city officials'
         | names and email addresses, the perfect base to set up a social
         | engineering attack. I really don't understand why all those
         | GitHub issues need to be publicly accessible like this, all it
         | takes is one team member to inadvertently post something
         | sensitive and you're stuffed.
        
           | prepend wrote:
           | City official names and emails are usually public records and
           | are in Boston. So it's no more of a risk than current levels.
           | 
           | There is a risk of posting the wrong info, but that should be
           | acceptable given the unlikely probability of that happening
           | and the harm of when it does. Posting a citizens form is
           | really bad, but posting meeting minutes probably isn't too
           | bad.
           | 
           | I think it's important to consider the risk vs the benefit.
           | How much is having better collaboration and avoiding a JIRA
           | license worth?
        
       | tantalor wrote:
       | > Get rid of paper forms... moving those forms online ... made
       | government services much more accessible
       | 
       | Not everybody uses computers. Not everybody has internet access.
       | 
       | Shouldn't paper forms still be available as a last resort?
       | 
       | I'm okay with "online first" but not "online only".
        
         | kgin wrote:
         | Government can never be anything-only. About 20% of adults in
         | the United States are considered functionally illiterate and
         | 10% are almost entirely illiterate. There will always need to
         | be the option to speak to a human who can assist you.
         | 
         | https://nces.ed.gov/datapoints/2019179.asp
        
         | mrtksn wrote:
         | > Shouldn't paper forms still be available as a last resort?
         | 
         | No, not really. The edge cases are easily resolved by helping
         | out those who cannot use the standard method. A friend, a
         | random stranger or a clerk would gladly fill an online form
         | when someone is having troubles.
         | 
         | Upkeep of a parallel infrastructure is hard and expensive.
         | 
         | If it's not THAT rare it instantly becomes a service that is
         | provided by a shop close to the place where the papers were
         | supposed to be filled.
         | 
         | I've seen that in Bulgaria where clerks would fill online or
         | paper forms for you in exchange of a small fee and in Turkey
         | there's a bookstore close to official buildings that will help
         | you with all kind of form. There used to be people with
         | typewriters hassling around public offices that would fill
         | forms for you too.
         | 
         | Nowadays in Turkey the online government services are very
         | good(you can do everything online, including everything from
         | health services to subscription cancellations), so these seized
         | to exist and if there's some who cannot use the online stuff, a
         | local shop or the clerks in a public office would help them
         | out.
        
           | cratermoon wrote:
           | psst: https://news.ycombinator.com/item?id=26512281
        
             | mrtksn wrote:
             | Interesting edge case that can be solved by booklet or a
             | person with half an hour time explaining how these things
             | work or simply help them out instead of upkeep of a paper
             | processing infrastructure through institutions.
        
               | woodruffw wrote:
               | My parents, who have decidedly _not_ been in prison for
               | the last 30 years, can barely fill out most digital forms
               | (especially ones on the Web) on their own. They both have
               | professional degrees and have used computers since the
               | 1980s.
               | 
               | Assuming that someone who has likely never touched a
               | computer before can operate at or above that level seems
               | like a fast track towards ensuring that they remain
               | marginalized and even further isolated from social
               | services. That, in turn, has long term expenses that, I'd
               | expect, dwarf the costs of a few hardcopies and a data-
               | entry clerk.
        
               | cratermoon wrote:
               | "edge case"? Take a look at how many Americans are under
               | the control of the carceral system. Dismissing the
               | reality of how we treat prisoners in the US smacks of
               | elitism.
        
           | blendergeek wrote:
           | I am only okay with online only if all online forms strictly
           | stick to open standards and protocols and require no data
           | collection besides what is strictly necessary for the forms.
           | 
           | When I have to agree to a "Terms of Service" on a third party
           | website to get my drive license and these Terms have a
           | mandatory arbitration clause and leave this private entity
           | with the ability to ban me for any reason, I will
           | understandably be upset.
           | 
           | All government forms must be accessibly to all. If internet
           | is required to use government forms, internet access must not
           | be cut off to anyone without due process of law. The same
           | goes to electricity and the ability to use electronic
           | devices.
           | 
           | If the main computing devices available are Google
           | Chromebooks, Apple iPads, and Microsoft Tablets, the user's
           | ability to log in can be restricted at the drop of a pin by a
           | faceless corporate entity. While it may still be possible to
           | access the internet on some of these devices without agreeing
           | to the TOS, this can change. If the government certificate
           | expires, then the device will need to be updated. Updates
           | almost always require a TOS agreement.
           | 
           | No private corporations should be able to ban people from
           | using government forms without due process of law.
           | 
           | So yes, I am okay with online only, but only in a world where
           | all software can be used by all users for any purpose without
           | threat of revocation from extra-judicial private overlords.
        
         | CityOfThrowaway wrote:
         | It is a significant reduction in scope and cost if you:
         | 
         | 1. Make it online only
         | 
         | 2. Let people use computers for free at city hall and libraries
         | 
         | 3. Have folks around who can help if they get stuck
        
         | thechao wrote:
         | Paper forms are available ... where? In my county, the only
         | place to get a paper form is by printing it out. A lot of
         | people use the library to get access to a computer ... to print
         | the form.
         | 
         | I'm not saying "no" to paper, I'm just talking about the
         | reality that having a computer in at least one part of the
         | process is mandatory.
        
         | prepend wrote:
         | Kiosks at city hall and libraries help with this, but I think
         | the simplest way to support this is to have each online form be
         | printable (or have a form to request a paper version) and then
         | an address to mail it with a message like "online forms usually
         | processed in 1-2 days, mailing forms depends on postal service
         | and may take 4-6 weeks"
         | 
         | And just do data entry on the few paper forms that make it past
         | this and don't think about it.
         | 
         | This is contingent on being able to have an alternate renderer
         | that spits out pdf based on the form definition that should be
         | dead simple.
         | 
         | I work in an org with lots of forms. People were weird that
         | they wouldn't accept a screen that said "Prepend approved on
         | 3/18 at 1055" and would do weird things like print out the
         | screen and ask for signatures or say funny stuff like "the law
         | requires a paper signature." So the pdf killer equivalent added
         | a "view form" button that just displayed the old form, filled
         | in from the same record with "Digitally signed by prepend on
         | 3/18 at 1055" in the signature box. Seems stupid but worked
         | wonders. Some people would print that out and save it, but that
         | doesn't slow anyone down but the printer.
        
         | TotempaaltJ wrote:
         | I don't think anything about this article implies that there's
         | no non-tech way to utilize these services after this change.
         | Most governments have laws banning that anyway.
         | 
         | A quick browse of boston.gov shows at least all forms I could
         | immediately find have an in-person alternative listed on the
         | website.
        
         | jeffgreco wrote:
         | Boston has a number of ways to access online services,
         | including free public wi-fi, libraries, and even a mobile City
         | Hall that brings services directly to different neighborhoods:
         | https://www.nationalcivicleague.org/promising-practices/bost...
         | 
         | More accessible than requiring someone to go to a City Hall
         | basement!
        
         | raverbashing wrote:
         | Moving the forms online makes it easier even if the person
         | using the system is still the office clerk.
         | 
         | Sure, they'll have to fill the form themselves, but it is
         | already online. So it is one (main) workflow regardless of who
         | does it.
        
         | caseysoftware wrote:
         | 100% spot on here.
         | 
         | There are a significant number of people who use their phone as
         | the primary or even sole method of internet access. And that's
         | not just low income people but also young adults. When you
         | start counting homeless people who primarily use the library
         | (if open), you have another problem.
         | 
         | I've worked in various "digital divide" issues for almost 10
         | years. Online first is okay but we _can 't_ wipe out the
         | offline approaches without leaving large swathes of people
         | without any feasible options.
         | 
         | Innovate but make sure you're considering your constituents'
         | needs.
        
       | Phenix88be wrote:
       | The amount of GAFAM dependencies here is impressive. So Google
       | has all the dead certificate request made by Boston resident ?
       | And more ? I don't like this one bit. And the article didn't even
       | mention if they try to use something else, so I guess they didn't
       | even try.
       | 
       | From my point of view, paper is better than any Gafam
        
         | etripe wrote:
         | For those similarly confused as I was, Gafam seems to be the
         | acronym for Google, Apple, Facebook, Amazon and Microsoft.
         | 
         | I'm sure I would agree to Google and Facebook being bad in this
         | situation. I'm mostly convinced about Amazon, too, but why
         | Apple and Microsoft?
        
       | ramboldio wrote:
       | > We've got a lot of users and a lot of submissions flowing
       | through SeamlessDocs. They are a great tool, but our contract is
       | structured in such a way that makes continued expansion tough.
       | 
       | I feel like an Open Source version of SeamlessDocs would be
       | incredibly useful to tackle these issues not just in the US.
       | 
       | Are there people here who would be interested in building
       | anything like this? Let me know.
        
         | impostervt wrote:
         | I'd love a product that is just "drag a pdf into a window and
         | we'll turn it into a form". It looks like seamlessdocs does
         | that, and then lets you edit the resulting form. But they're
         | focused on governments, and I still have to fill out stupid
         | forms for my company on paper (usually health care or tax
         | documents or paycheck stuff) because they send THOSE on to
         | whichever company we've outsourced that work to.
        
       | bottled_poe wrote:
       | It's not even my final form
        
       | jackconsidine wrote:
       | At my company we use Github Issues for all task management. Like
       | this author discovered, it is underrated.
       | 
       | SaaS products like Asana and Trello have prohibitive pricing
       | models for what we consider pretty minor integrations. Jira is
       | too complicated. I have seen some open source projects recently.
        
       | devtul wrote:
       | > Dogs (can this be a drop down as well from 0-9)
       | 
       | Ooh boy, I wonder how long the drop down for cats would be.
        
       | wiz21c wrote:
       | FTA: That sort of works, but it also creates a weird dynamic
       | because the tech teams don't have the training nor the explicit
       | mandate to mess around with other people's jobs.
       | 
       | Here in Belgium, I did that kind of work for 14 years. The fact
       | is that you have to have in your IT team people knowledgeable
       | with business. When I say knowledgeable, I mean, they understand
       | the business at least as well as the business people themselves.
       | When you talk to business with that kind of team, it's much
       | easier.
       | 
       | Now, the problem I had is not the form, it's the business process
       | behind. Changing these is super hard because inertia is there.
       | Also you have to take into account years and years of little
       | shortcuts, exceptions, etc. that somehow now are part of the
       | process. It's really tough to bring everything together.
        
         | Ajedi32 wrote:
         | Reminds me of this interview with Elon Musk, where he talks
         | about Conway's Law and how they mitigate it at SpaceX:
         | https://youtu.be/cIQ36Kt7UVg?t=206
         | 
         | It's very hard to design an efficient system when the people
         | writing the requirements and the people implementing them don't
         | really understand or talk to each other.
        
       | motoboi wrote:
       | > I put in my earbuds, fired up a podcast, and started going line
       | by line through the list. That took about 2 months. I would open
       | the PDF, see if it was a form, and log it.
       | 
       | You can see this guy is not a developer because he didn't spend 2
       | months learning machine learning and computer vision while trying
       | to automatize the form classification task only to give up after
       | 6 months and do it by hand anyway.
       | 
       | And clearly government because he didn't spend 10 dollars in
       | mechanical turks to do it in 24 hours.
        
         | josefresco wrote:
         | Form Yes/No certainly but mechanical turks can't handle this
         | part:
         | 
         | "I also reviewed contracts and documentation and interviewed
         | employees in the various city departments to ask if they had
         | other forms."
        
       | notjustanymike wrote:
       | Impressive credit to the author for writing an interesting
       | article about the least interesting topic imaginable: government
       | bureaucracy.
        
         | Angostura wrote:
         | But it's not about government bureaucracy - it's about making
         | both the lives of your coworkers and your fellow citizens just
         | a bit better. That can actually remarkably fulfilling.
         | 
         | I'm working with a group of doctors practice in London on their
         | digital plans at the moment, and I love it.
        
       | dsego wrote:
       | For administrative fees maybe there could be a digital version of
       | the tax stamp?
        
       | codingdave wrote:
       | The entire landscape of this problem changed this year -- The
       | latest version of Word now can open PDFs and edit the content,
       | then save back to PDF. PDFs are no longer roadblocks, they are
       | now a document type that just works.
        
         | jdsnape wrote:
         | only if you can afford to buy word, surely?
        
           | codingdave wrote:
           | True, but most government entities I work with use Word as
           | their primary authoring tool, so that isn't a major barrier.
        
             | munificent wrote:
             | I think the people filling these forms out are generally
             | citizens of Boston, not city employees.
        
       | ccleve wrote:
       | Well-written article, useful insights, total misunderstanding of
       | the way things like this should be done.
       | 
       | There's an old saying: don't pave cow paths. That's exactly what
       | he was doing. He was putting a different front end on an existing
       | process. You end up with a total nightmare.
       | 
       | Instead, this should be done department-by-department, app-by-
       | app. Design the process from the bottom up. For example: a form
       | to request some document. Start with the documents: how are they
       | stored? Should they be stored differently? Are they in a
       | database? Should we really even be storing documents, or should
       | they just be database entries? For that matter, should the
       | department that manages them even exist? (Here in Chicago, the
       | answer was no. They merged the County Recorder of Deeds office
       | with the County Clerk.)
       | 
       | You should never just move things online. The web is different,
       | and enables different business processes. Change the process
       | first. Forms come later.
        
         | techbio wrote:
         | The iceberg image at the top of the article fails to show how
         | much of an actual iceberg is beneath the surface.
        
         | Spooky23 wrote:
         | I have expertise in the space and while I would have agreed
         | with you 10 years ago, I disagree today.
         | 
         | There are billion dollar programs that can be replaced by a
         | Google Form. You're always going to lose if you are too
         | accommodating. Every one of these dopey workflows has a unit of
         | FTEs on payroll in the backend!
         | 
         | Get High level sponsorship, hard deadline and most things will
         | be doable within reasonable constraints. The stuff that isn't
         | doable is the hard work, which you will never discover until
         | it's forced.
        
         | CityOfThrowaway wrote:
         | This approach is likely to produce better experiences if it
         | could be seen through to the end, but also likely to go nowhere
         | inside a large organization.
         | 
         | Change is hard. Radical change, all at once inside a big
         | organization is near impossible.
         | 
         | In this case, it's a great first step to get all this info out
         | of paper and wooden inboxes and into a more accessible form.
         | 
         | Doing so still leaves open the potential for reinventing high-
         | priority forms and workflows. It also makes it easier, because
         | now the team can easily ask questions like, "how many people
         | use this?" And "how much time do we spend on this?" Etc.
         | 
         | In short, while you might be right wrt. producing innovation,
         | sometimes the first step is to just put it on the internet.
        
         | williamsmj wrote:
         | > Instead, this should be done department-by-department, app-
         | by-app. Design the process from the bottom up."
         | 
         | Sounds great. If you can do this in less than a week for each
         | form (the author managed 100 forms in 2 years) then you're
         | perfecting processes _and_ moving them online faster than the
         | author.
         | 
         | But just in case you missed it, let me quote a couple of bits
         | of the article:
         | 
         | "Getting city workers to accept online submissions rather than
         | traditional paper ones is the bulk of this work. On average, it
         | took me about 30 minutes to make a digital form and five weeks
         | to meet with, earn the trust of, and get buy-in from the
         | employees who would use it. Even if they were excited, the
         | nitty gritty details took a lot of back and forth."
         | 
         | "Some departments had sort-of insane business processes for
         | submissions. If I tried to change those, I would spend a whole
         | year on a single department. By focusing on the priority,
         | moving forms online and making it easier for the customer, I
         | could make consistent progress rather than be consistently
         | blocked."
         | 
         | My guess is that you're going to spend decades perfecting
         | processes, only to find the world has changed (or the
         | politicians in control of the City of Boston have changed the
         | law that governs the processes!) by the time you're ready to
         | begin digitize forms. You're letting perfect be the enemy of
         | good.
         | 
         | > You should never just move things online. The web is
         | different, and enables different business processes. Change the
         | process first. Forms come later.
         | 
         | Sometimes "just moving things online" is a big improvement over
         | the status quo. But the author agrees with you in at least one
         | case. See the paragraph about what they are doing about death
         | certificate requests.
        
           | fluidcruft wrote:
           | Yeah, I agree. A lot of what I deal with at my job is
           | similar. The people doing the work just don't want people
           | coming in telling them new ways to do things. It doesn't help
           | that vendors are constantly promising the sky and leaving
           | them with broken things. But once you start showing them
           | small incremental improvements they start to discover the
           | next step etc. Then they see how machines can help them and
           | make suggestions.
           | 
           | In my field there's a very well-known example of a MRI
           | manufacturer who has made very "smart" systems that end-users
           | simply refuse to use. Technologists know how to operate a
           | scanner, when a machine starts trying to help them it's very
           | difficult for them to (1) understand what the machine is
           | doing and how to work with it and (2) trust the machine,
           | primarily because physicians yell at people, not machines.
           | There are maybe 10-20% of the technologists that "get it" and
           | want to tinker and learn how to use the new features. The
           | rest still miss the way scanners worked 20 years ago--because
           | they understood and trust them.
           | 
           | That's the thing with paper forms--people can understand
           | what's happening, discover the process and figure out how to
           | succeed and not to be yelled at. When it disappears into a
           | machine it might as well be a blind and deaf new hire who
           | keeps messing with your desk. It's a major issue of trust.
           | 
           | So the parent's "no it needs to be this giant reinvention
           | when strangers who don't do the work come in and tell you
           | that you need to do everything differently" attitude maybe
           | works with management.
           | 
           | I honestly think the big problem here is that we don't seem
           | to have systems that people can play with to build their own
           | ideas anymore. Maybe sharepoint but my god I can't even
           | figure out how to do anything on that. It really boggles my
           | mind that we have things like iPads and iPhones but nothing
           | like Hypercard.
        
         | dragonwriter wrote:
         | > Design the process from the bottom up
         | 
         | Bottom up _IT system_ reimplementations fail much more often
         | than incremental ones.
         | 
         | Bottom up reimplementations of operating _human_ systems that
         | _also_ involve big bang IT implementations are even worse, by
         | comparison to incremental evolutions, mixing incremental
         | automation with incremental process improvement.
         | 
         | And it usually only takes one high-profile failure of a big-
         | bang implementation to derail a broad reimplementation process,
         | whereas the occasional incremental setback on an incremental
         | improvement process is rarely politically significant to the
         | overall process.
         | 
         | So, no, I disagree with your recommendation that this should
         | instead have been done with a more waterfall-style approach.
        
         | watwut wrote:
         | > For that matter, should the department that manages them even
         | exist?
         | 
         | Yeah, nothing better then complete reorganization by someone
         | who has no idea about the work being done, needs and pressures.
         | 
         | What will happen here is that they will quickly realize you
         | dont know what you are doing but are threat, so whoever will
         | have option will easily stall you.
        
         | sputr wrote:
         | Except what he did resulted in progress.
         | 
         | What you are proposing most of the time will not because it
         | would require an immense, truly Herculean, amount of political
         | will and backing.
         | 
         | Redesigning processes clashes with the #1 rule of big systems:
         | processes are ruled by people whose jobs depend on them. And
         | they WILL fight tooth and nail to keep those jobs. The bigger
         | the change, the more threatened they will feel, the more they
         | will sabotage you.
         | 
         | And you know what the #2 rule of big systems is: no matter what
         | "the boss" may say, if the officials are not on board it's just
         | not going to happen.
         | 
         | If you have not, I suggest watching "Yes minister" :).
        
           | watwut wrote:
           | Sure. But also, lets admit that software developers are not
           | qualified to redesign processes of institutions they dont
           | understand, so fighting them is pretty often rational from
           | institutional point of view.
        
             | 908B64B197 wrote:
             | If the processes of an institution can't be explained to a
             | real, qualified software engineer, the issue runs much
             | deeper than just resisting to change.
        
           | ghaff wrote:
           | People don't like to be told they have to change the way they
           | do things because... the computer.
           | 
           | Years ago, I was at a very small company that used Exchange.
           | We had some outages and it was just a lot of administrative
           | overhead in a world in which Gmail was by then an option.
           | 
           | But one person in particular fought the change strongly
           | because they were used to a business/sales process that used
           | hierarchical folders to keep documents organized and Gmail,
           | at the time, only had single level labels. The CEO mandated
           | the change anyway but the other person never accepted that
           | Gmail was suitable for our needs.
        
         | MattGaiser wrote:
         | > Change the process first. Forms come later.
         | 
         | In government that can easily mean the forms coming never.
        
           | cratermoon wrote:
           | That's just as true for the corporate world. Check out the
           | business processes of, say, a global telecom that's been
           | around since the era of party lines. SAP SE makes buckets of
           | money from big corporations that just want computerized file
           | cabinets.
        
         | ziziyO wrote:
         | Wasn't one of the the points of the article the complete
         | opposite of this comment? You can improve the process without
         | having to overhaul the whole thing. They don't need to flip
         | their tech stack for forms.
        
         | spelunker wrote:
         | The author was given the mandate to "move forms online", so
         | they chose the quickest, most agile option and did just that -
         | moved forms online.
         | 
         | The author later brings up the questions you mentioned - what
         | is a form, should these forms exist, etc, but instead of
         | spending a long time trying to design an entire new system
         | _when that wasn't even asked of them_ they decided to deliver
         | results. Once they gained trust and got buy-in by the city,
         | they could think bigger.
        
         | codeulike wrote:
         | Except to redesign the processes you need buy in and co-
         | operation from multiple different people and departments and
         | good enough project management and communication with all of
         | them to ensure that the thing you replace the forms with will
         | actually do the same job.
         | 
         | Whereas moving existing forms online in the same structure can
         | be done with minimal management agreement and provides a quick
         | win.
         | 
         | Especially when you're talking about 344 forms. Redesigning all
         | the back office processes for 344 forms would be a gargantuan
         | task.
        
         | corwinstephen wrote:
         | This is 100% correct. I run a govtech startup (similar to
         | Seamless docs, the company mentioned in the article), and this
         | was one of the key insights we've picked up over the last
         | couple years.
         | 
         | When your process happens on paper, there's nothing forcing you
         | to ensure it makes logical sense. You can draw whatever you
         | want on a piece of paper, so a lot of the paper forms that
         | people call "permits" are just messy reflections of the thought
         | process of the person who originally conceived the process.
         | Naturally, products that start by directly replicating a paper
         | form are accommodating and perpetuating the lack of planning.
         | 
         | We've kind of taken a backwards approach where we don't
         | accommodate direct translation of forms and instead provide
         | tools that encourage people to think through the steps of their
         | permitting processes to get them to make sense on a digital
         | platform. More often than not, they discover glaring
         | inefficiencies in the process which when fixed make everything
         | work more smoothly even independent of the move to a digital
         | platform.
         | 
         | Shameless plug: https://citygro.ws
        
           | [deleted]
        
         | at_a_remove wrote:
         | I will disagree. In the distant past, we had a bunch of forms
         | to move online and our platform was SharePoint. I know, I know,
         | but before you blame SharePoint, we didn't get to the point of
         | SharePoint being the issue.
         | 
         | Instead, we got endless bikeshedding about "the process" and
         | during that, many unrealistic expectations were conjured into
         | existence as each process was re-imagined into kind of
         | ultimate, abstract form of near infinite configurability. Try
         | to imagine sitting at a table as someone says, with complete
         | seriousness, that people _ought_ to be able to submit a
         | multiple one terabyte (not mega or giga, tera) files with a
         | particular webform. Not any other transfer protocol, either.
         | And this is back around 2008 or so. It should just ... _expand
         | as necessary_.
         | 
         | Nor was this the only time I encountered this kind of behavior.
         | I have seen functionaries whose visions of process suddenly
         | bloat like that "elephant's toothpaste" when presented with
         | technological options. I would do the forms first, given the
         | chance.
        
         | specialist wrote:
         | > _...the way things like this should be done._
         | 
         | Do you have case studies for successful process reengineering
         | efforts?
        
         | avmich wrote:
         | Did you try to do something like that - moving a government
         | office into digital world? What is your personal experience
         | with this?
        
         | pc86 wrote:
         | Your misunderstanding of how things work in government is at
         | least as egregious as the author's misunderstanding of this
         | idealized way to redesign processes.
        
         | hellohello1 wrote:
         | Great idea! I wonder why nobody has ever thought about that
         | before! You must be the first! A modern day Prometheus!
         | 
         | The highlighted part about the fact that it takes 30 minutes to
         | make a change, and FIVE WEEKS to get sign-off is spot on. I
         | fucking hate bureaucracy. But you know what happens if you
         | don't take the FIVE WEEKS to get sign-off? Nothing.
        
         | ordu wrote:
         | _> Instead, this should be done department-by-department, app-
         | by-app. Design the process from the bottom up._
         | 
         | It is a technical way. "Do not mind people, just make tech to
         | work". This way have its benefits, like it tends to make result
         | to have better design, but what about people? They work and
         | they think that their work is important and valued. But "hey,
         | some engineer came and started to make it his way without any
         | reverences to us". It is humiliating, and such engineer could
         | face a resistance.
         | 
         | The author made a genius' move which I might expect from a
         | psychologist or any other social oriented person, not from
         | tech-savvy engineer: he started to make life of a people
         | better. He asked their opinion on how they think it would be
         | made better. He showed respect. He showed people that he sees
         | them as people, not as gears in a clockwork mechanism. They
         | reacted as people generally tend to react to this: they
         | welcomed him. Probably it helped that it was government
         | bureaucracy which likely treat people as gears in a clockwork,
         | so his way of treating people contrasted nicely to the way they
         | used to.
        
         | tokipin wrote:
         | It seems more like a classic case of refactoring except that
         | the original infrastructure is paper-based, in which case the
         | ideal digital redesign would be one that mimics the paper-based
         | one precisely so that "regression testing" can be performed and
         | components can be swapped in piecewise. Once enough components
         | are digitized, the system as a whole can begin to be molded
         | into more effective forms. This would have to be the plan at
         | the outset, since it would require preserving enough
         | flexibility.
        
         | gregd wrote:
         | A lot of government agencies lack the budgeting required to go
         | through a process of "as-is" and "to-be". This process is both
         | lengthy and costly and is usually reserved for the main portion
         | of that agencies workload.
        
         | motoboi wrote:
         | I'd love to read more about this view. Do you know any good
         | books about this subject (government digitalisation)?
        
         | saul_goodman wrote:
         | You're not technically wrong, but what you are suggesting
         | simply doesn't work in government. Each department is a kingdom
         | with varying amounts of political clout. Only the smallest and
         | least-powerful groups will jump on board with a mandate that
         | induces invasive and drastic changes to their workflows. The
         | kingdoms with more political clout will laugh in your face and
         | slow your progress down until your project simply runs out of
         | clock time. The result is that you still have all the same
         | legacy workflows and also this new workflow for the small
         | number of groups you had more power over.
         | 
         | This guy managed to achieve his mandate AND managed to entice
         | groups to investigate updating their workflows after the fact.
         | I agree it's not the way things SHOULD work, but he managed to
         | find a way to make the most amount of progress and along the
         | way some kingdoms were enticed to cycle back around and improve
         | their workflows too.
        
           | m-watson wrote:
           | In DC their in house 'innovation team' tried to overcome some
           | of these issues and make them more useful to the people using
           | the forms by having an annual formapalooza. Results were
           | mixed but it is an interesting approach to digitization and
           | human centered design.
           | 
           | https://thelabprojects.dc.gov/formapalooza
        
           | drc500free wrote:
           | Great point. With the Federal Govt that often breaks down to
           | "agencies that use GSA as a service provider" and "agencies
           | that roll their own."
        
           | OminousWeapons wrote:
           | I haven't done a lot of work in government, but I have done a
           | lot of work in orgs with distributed power structures, and
           | this is 100% correct. Starting from the ground up means
           | constructing a working group for every single workflow in
           | which everyone who is even theoretically a stakeholder (and
           | many people who aren't) is going to demand to be in the room
           | and be heard. Once you get past 4-5 people, good luck holding
           | a meeting more than once every few months, and if a single
           | powerful stakeholder doesn't like what you are doing or
           | perceives that you are stripping them of power, they will gum
           | up the process until it fails. I've had 4-5 value adding
           | initiatives crushed in this manner at past jobs.
           | 
           | IMO, the optimal strategy in such an environment is to get at
           | least one powerful champion on your side who can give you
           | political cover, and then to quietly construct a MVP while
           | soliciting feedback from high performing rank and file in the
           | departments of interest who you know (ie no other power
           | brokers besides your champion(s)). If you can deploy a value
           | adding MVP it becomes difficult for antagonists to intervene
           | and kill the project without appearing petty, and then you
           | can really get to work building the product you want.
        
           | m463 wrote:
           | I sort of imagine some departments opting for a webform-to-
           | fax gateway :)
        
         | droopyEyelids wrote:
         | Do you see any middle ground between the 'rebuild everything
         | from the ground up' and 'rework the user interface' type
         | approaches?
        
         | jimmaswell wrote:
         | That saying is strange because cow paths usually represent the
         | most direct path somewhere that the designer of the space
         | failed to imagine.
         | 
         | Is this picture a bad thing?
         | https://i.redd.it/uj5ehfdxn5o31.jpg
        
           | floatrock wrote:
           | That saying is also oddly appropriate here because the joke
           | for anyone who lived in boston is that the city is built on
           | paved-over cow paths.
           | 
           | (yes yes fun-police, it's actually a myth
           | https://www.bostonmagazine.com/news/2018/03/06/boston-
           | street... but the truth of it being a bunch of ad-hoc cities
           | that coalesced into one entity isn't too far off from the
           | truth. My favorite bit of Boston artwork is road silhouettes
           | of all the Boston-area squares:
           | http://andywoodruff.com/blog/boston-squared/ )
        
           | cratermoon wrote:
           | Those aren't cow paths, those are people paths. Cows are not
           | known for their tendency to walk in straight lines. Edit:
           | notice they still had to put up fencing to keep people from
           | cutting off even that minimal switchback.
        
             | frenchy wrote:
             | This might be a dialect issue. Where I'm from, "cow path"
             | is the normal way of saying what also called a "desire
             | path" by others. According to the wikipedia, I'm not
             | completely out to lunch:
             | https://en.wikipedia.org/wiki/Desire_path
        
               | jimmaswell wrote:
               | I hadn't heard "cow path" before, but a quick google
               | suggested it was a synonym for "desire path" so I went
               | with that.
        
           | frenchy wrote:
           | It is a strange saying. My guess is this is something that's
           | lost in translation--a leaky metaphore--and the people using
           | it don't think about real cow paths.
           | 
           | Real world space is (generally) 2 dimensional, and these cow
           | paths are often very predictable and sensible. They're
           | usually the result of architects thinking that sometimes
           | people actually walk or bike, instead of always driving cars.
           | 
           | In the world of bureaucracy & information tech, the space can
           | be very convoluted, and shortcuts aren't really the same kind
           | of thing (queue XKCD comic on spacebar heating).
        
         | jimmaswell wrote:
         | The argument in the article for gradual change sounded
         | reasonable to me. Just moving a process online is at least a
         | first step.
         | 
         | > Some departments had sort-of insane business processes for
         | submissions. If I tried to change those, I would spend a whole
         | year on a single department. By focusing on the priority,
         | moving forms online and making it easier for the customer, I
         | could make consistent progress rather than be consistently
         | blocked.
         | 
         | Re: should they just be database entries? he did do this or
         | make other changes when the department agreed.
        
       | jimmaswell wrote:
       | Sometimes the best form is no form at all. The Washington state
       | license/registration process was incredible, just show up with
       | the documents and they ask some questions and type the relevant
       | stuff in for you. Compare to the ridiculous waste of time and
       | paper in NY where you have to arrive with about half a dead tree
       | of redundant information that they just retype into a computer
       | anyway.
       | 
       | As a more general point, living in multiple states shows you how
       | much bloated government is a complete waste of time when other
       | states just don't do it and get along fine - other examples are
       | car inspections and even income or sales tax.
       | 
       | Did the city decide to simply eliminate any of the functions the
       | forms you found did instead of moving them online?
        
       | DiggyJohnson wrote:
       | Fantastically written article. Fantastic article. It's always
       | great to have both.
        
       | matsemann wrote:
       | This is the first step of digitization: Putting electricity on
       | the paper. But it's only the beginning. For the long-tail it may
       | be good enough. But for lots of government forms there are so
       | much more that can be done.
       | 
       | Like, we discovered some absurd amount of forms sent to us (like
       | 90% or so) had errors that actually would make the applicant
       | ineligible for whatever the form was for. Stuff like "if you
       | answered X on Q14.2c, you have to answer Q32.5". All this is
       | something good UX and logic can help make a breeze. Instead of a
       | huuuge form to cover all edge cases, it only expands when veering
       | from the happy-path.
       | 
       | Also, next step of digitization is actually getting the data as
       | structured data. So that it's automatically put into whatever
       | software the govemp uses to handle the form. Without that,
       | majority of the amount of time spent from a caseworker can be to
       | just transcribe the form into application before it's actually
       | being worked on. Next step after that is of course to
       | automatically approve/deny stuff. If not for everything, there
       | are lot of simple cases that often don't need much scrutiny.
       | 
       | A big step is to actually get the gov to change the process/law.
       | Putting electricity to paper is a small win over the incredible
       | wins with just fixing the process. Like one form I worked on was
       | so complicated (maternity/paternity leave) because the law had
       | basically hundreds of small edge cases accrued over the years.
       | Either to fix a loop-hole, or to help some group of people a bit
       | extra. But that just ends up in regulation not even the experts
       | approving these forms understand. It's a hard battle and can take
       | years. Like, sometimes a new rule the politicians invent will
       | cost far more to implement and execute for the small case, than
       | just making it apply to everyone and grant them some extra
       | money...
        
         | zentiggr wrote:
         | All very valuable and extremely on point for the person/team
         | who will be taking over the process after the OP.
         | 
         | I've had my own very small share of experience delving into
         | existing workflows and teasing out the early/simple cases and
         | only getting into further processing once the basic conditions
         | are met. It can be an interesting challenge to get those who do
         | the current processing to see where efficiencies can be teased
         | out.
         | 
         | The level of legal/higher management mandates for significant
         | process changes is something I've never had to deal with, but
         | I've seen the edges of just how crazy the complications can
         | get. (I once did an implementation of Form 1040 handling via
         | spreadsheet... I was surprised to find that proper math
         | REQUIRED a circular reference and iterating to correct
         | numbers.)
        
       | Bukhmanizer wrote:
       | Great article. It felt true to my experience working with govt
       | employees, and I'm impressed that this person had the tenacity to
       | achieve the success that they did.
       | 
       | Unfortunately I also think this is the kind of stuff its really
       | easy to write a cheap headline about.
       | 
       | "BOONDOGGLE: BOSTON SPENDS $150,000 AND 2 YEARS MAKING PAPER FORM
       | INTO WEBSITE"
       | 
       | There, that's 100,000 shares on a headline that no one will
       | really care to read the details of.
        
       | known wrote:
       | In my experience it's e-Redtape :)
        
       | ilikerashers wrote:
       | What I've seen continually in government is the dreaded form-
       | builder Frankenstein.
       | 
       | 1. Gov thinks it needs some simple form building 2. Chooses a
       | vendor form building framework, some mashup of ASP.NET usually
       | with workflow tools 3. Oh, we'd like documents stored, and an
       | integration with another system, and a custom reporting tool, and
       | a payment function, and a appointment booking. 4. Vendor doesn't
       | want to lose business so builds custom code into the "simple"
       | form/workflow tool.
       | 
       | All of sudden the simple form tool is customized beyond belief,
       | falls behind on updates and becomes impossible to change. Worst
       | still, nobody wants to invest in it because it just-about-works.
        
         | spaetzleesser wrote:
         | I work in medical and there the top guys often make decisions
         | that are really hard to implement in software. If they
         | understood only a little of the consequences of their decisions
         | have they would probably think a little more about the rules
         | they are creating.
         | 
         | I see the same in product development. Marketing comes up with
         | something that has words like "most of the time", "sometime" or
         | "somehow" in the description. Really hard to impossible to
         | implement. When I ask them what they really want to achieve we
         | usually end up with something simpler and straightforward that
         | fulfills the needs, is well understood and easy to implement.
         | 
         | I think in government with most of the work being outsourcend
         | this feedback loop doesn't exist. A committee comes up with
         | something and the contractor implements it without questions
         | asked. The more complex the work, the more money for the
         | contractor.
        
           | cratermoon wrote:
           | I work with authentication protocols and access control.
           | Product people want magic.
        
             | spaetzleesser wrote:
             | I am fine with product people expressing their wants. But
             | we shouldn't accept their requests blindly but put them
             | through a reality check. The customer is not always right
             | or at least the customer doesn't explain his real problem
             | properly.
        
               | cratermoon wrote:
               | That's true, but power dynamics don't always align with
               | wishes. If the professional recommendations of security
               | people weren't so routinely ignored, you'd see a lot less
               | stupid breaches. A great many disasters^1 could be
               | prevented if experts weren't ignored.
               | 
               | [1]
               | https://calteches.library.caltech.edu/3570/1/Feynman.pdf
        
         | stopyellingatme wrote:
         | Nail on the head.
        
         | mattmanser wrote:
         | ASP.net with windows workflow foundation hasn't existed for
         | like 10 years or something like that.
        
           | wernercd wrote:
           | VBA hasn't been updated since 2010... yet it's still going
           | strong.
           | 
           | Things like this create a niche and will live longer than
           | cockroaches after a nuclear war.
        
             | mattmanser wrote:
             | VBA is still the main way of writing macros. There are
             | plenty of jobs requiring VBA.
             | 
             | Try and find a job for asp.net webforms and windows
             | workflow foundation.
             | 
             | So kinda pointless you posting that really. One is still
             | 'current', but the other you'll be writing asp.net core
             | with modern MVC.
        
       | cube00 wrote:
       | It's scary that they're now locked into SeamlessDocs which has
       | stopped publishing its pricing.
       | 
       | 2016:
       | https://web.archive.org/web/20160303073934/https://www.seaml...
       | 
       | Today: https://www.seamlessdocs.com/pricing
        
         | prepend wrote:
         | This is what makes paper forms so hard to crack. Paper and PDF
         | is a predictable price that will be about the same in 10 years,
         | 20 years, whatever.
         | 
         | Not knowing what SeamlessDocs charges (or will even be around)
         | is hard to plan around. They do call out a new contract that I
         | think is meant to help with this.
        
         | mattmanser wrote:
         | Perfectly normal for this sort of product to not have fixed
         | tiers, they've just pivoted off trying to sell to self-serve
         | customers, to switching to a different pricing structure.
         | 
         | You can't make money like that if you have to negotiate
         | contracts with government agencies.
         | 
         | Not sure how that's scary.
         | 
         | Imagine sending in a salesperson, and occasionally an
         | developer, to 5 meetings, each lasting 2 hours, over the space
         | of 6 months. Now calculate how long it would take to pay for
         | that back at $55 p/m.
         | 
         | And you'll note that the page already had an "enterprise" tier,
         | for which you had to enquire.
        
       ___________________________________________________________________
       (page generated 2021-03-19 23:01 UTC)