In 1985 a well-known publication in 1985 reported on a retrieval experiment where lawyers searched for legal information. They found only about 20% of the potentially relevant documents. The authors concluded that full text search has serious limitations.
In a succeeding article, the publication was criticized by Gerald Salton, a professor at Cornell University. An essential aspect of the debate was about set-oriented retrieval as opposed to ranked retrieval. Today, set-oriented search term reports are still widely used in court cases while ranked output is adopted by internet search services such as Google and Bing. Internet search engines use Artificial Intelligence (AI), in particular machine learning, as the search quality is significantly better. However, modern approaches are less transparent and more difficult to verify.
To understand how such machines learn and what they eventually learned from the training samples is complex. The dependency between search quality and training material is still subject to ongoing research. On the other hand, simple search term reports are easy to understand and verify. In summary, there is a trade-off between search quality and verifiability.
