This Search Auditing team at a major tech org. has a need for a group of Super Auditors for the Local Search relevance project that would work closely with the team lead to produce the resources needed to boost the overall quality of the off-site judge resources.
In other words, you are the human element of search, balancing the work of the computer algorithms, and making sure that what the computers are doing makes sense to real people! We are looking for One International resources - A Native Great Britain English Speaker from Great Britain is nice to have, but not required. More important to have the experience with auditing, and strong research experience.
Real-Time Audits (RTAs) test resources against definitive judgments, and provide simultaneous training on the correct approach to use. These are either created as an outcome of regular audits, created directly through UHRS tools, or created offline and uploaded to the system manually. In the tool, a small percentage of data is sent to at least two judges; those with disagreeing ratings are available for auditing by the Vendor. A single audit will count for all judges involved (typically two).
Spam Creation - This test needs to be created by the Quality Auditors to silently check judges against a set of definitive judgments.
Review Project Guidelines - The Auditor will need to review project guidelines from time to time & suggest improvements if any
The Auditor will be responsible for aggregating the Quality Data and Analyze the test results and make recommendation to improve Judge Quality