Stop Condition

The stop condition defines when reviewers can stop reviewing new documents and the iterative Assisted Review process is terminated. No new training batches will be added. You can select Classify Remaining (if not all documents have been reviewed already). The last trained classifier is used to classify the remaining documents.

Several stop conditions can apply. It depends on your project when to stop.

Stop Conditions

• The Recall Goal is reached.

  The Recall Goal is set when defining a project with a Validation Set.  

  In the Assisted Review Progress tab, monitor if the Recall Goal is reached or nearly reached (Estimated Current Recall):

  

• The marginal precision is below 10% (0,1) or higher than 80% (0,8). 

  The marginal precision is the precision of the last reviewed training batch added to the training set, also called the return set. The marginal precision usually rises to a high value (for example, 67%) in the beginning of the process.

  View the marginal precision in the Precision of Return Set graph.  

• The Gain Curve has reached its plateau.

  If the gain curve reaches a plateau, enough iterations/training batches have been done. The Assisted Review process can be stopped. 

• The curve of the Precision by Recall graph reaches a high precision and recall (0,8 or higher), or is not improving anymore.

• Precision and recall have both reached the values that were agreed earlier (for example, during meet-and-confer sessions).