Wednesday, December 7, 2011

Tax Experts vs. Robots?

In the battle between robots and legal experts, I have a stake in both camps. On the one hand, the site we're developing at Tabulaw, tax26.com, relies heavily on automated parsing of tax law information, in order to organize and display it intelligently for professionals.  On the other hand, this endeavor, and everything I have done professionally, require varying levels of expertise in many, usually esoteric, domains.  In fact, when my 5 year old is asked what her parents do, she says that her mom is a teacher and her dad is an expert.  Expert in what, none of us is quite sure.

So it should be no surprise that I am intrigued by the esoteric post about this battle by John Barker of Wolters Kluwer (a large legal publisher and owner of CCH, an information service for tax professionals). Barker argues that CCH's product, backed by legal experts who understand the context of arguments in a tax law case, is better positioned than Google Scholar to provide a meaningful result for professional users. Edward Bryant, one of those experts at CCH, writes on his personal blog that the tradeoff between expertise and automation is foremost a question of money: "(1) is an automated algorithm cheaper and (2) is the accuracy level I get through automation acceptable to my customers"?

I have understood these tradeoffs most clearly in discussions with Itai Gurari, formerly of Google Scholar, creator of Tracelaw, and one of the world's experts in automation of caselaw analysis.  What Itai does is really hard, and there aren't many people who can do it: while caselaw has an underlying structure due to its logical place in legal canon, finding that structure is an intrinsically hard problem to automate, because every judge thinks he's a poet, novelist or comedian.  True experts in a given area of caselaw are rare.  Experts who can formalize their knowledge into computer code are even more rare, so automation has a pretty large up-front cost.  And there are many subtleties that automated analysis will miss, no matter how many expert hours went into building it.

Where automation shines --or more broadly, computer assisted information processing-- is in the display and facilitated navigation of information.  Yes, experts' hand-drawn maps have been useful for a very long time, but even the Thomas Guide (remember those) has given way to Google Maps for most uses.

And that's where we're going with tax26.com: the tax map feature is just the tip of the iceberg of what can be done to map the geography of legal information. Not that this will replace experts any time soon, or my 5-year old will have to come up with a new title for me.