Ian Soboroff

Background

About Ian Soboroff

I build datasets to measure the effectiveness of search engines and other technologies for information access. As Kelvin said, if you can't measure it, you can't improve it. I manage the Retrieval Group at the National Institute of Standards and Technology (NIST). We organize the Text REtrieval Conference (TREC, http://trec.nist.gov) evaluations of information access, the Text Analysis Conference (TAC, evaluations of natural language understanding, and the TRECVID (http://trecvid.nist.gov) evaluations of video search. These are international collaborative research conferences focused on providing datasets, inventing measurement methods, and measuring the quality of information access systems. We also help design and support other evaluations in the broad area of access to information stored as natural language. I am a scientist who enjoys working on the full stack. I write code, gather data, run experiments, develop theories, implement systems, build hardware, and publish papers. Sometimes all at once. I am interested in measuring the quality of search, and in developing test collections for new or unusual search settings. Specialties: information retrieval, IR evaluation, test collection building and validation