Estimated Current Recall

The Estimated Current Recall of an issue is displayed in the Assisted Review Progress tab.

This will help you determine if the Recall Goal has been nearly reached and if you need to continue training the system, either by starting another training round or by extending the Validation Set. The Estimated Current Recall of an issue can only be calculated if you have added a Validation Set to your project.

 

Estimated Current Recall = #responsive found / #estimated responsive 

#responsive found = total number of reviewed responsive documents in the project (including the reviewed responsive documents of the Validation Set)

#estimated responsive = the estimated number of all responsive documents in the project 

The estimated number of all responsive documents in the project = total number of documents in the project * (#responsive found in Validation Set / (#responsive found in Validation Set + #not responsive found in Validation Set))

 

We perform the calculation with the numbers displayed in the graph below. The total amount of documents in the project is 175.000

Estimated Current Recall = (30.000 + 1.000) / (175.000 * (1.000 / (1.000 + 750))                                         

= 31.000 / 99.750            

= 0,31

                                      = 31%

An Estimated Current Recall of 31% is too low if the Recall Goal has been set to 80%. Create a New Training Batch or Extend Validation Set.

Note: If you extend the Validation Set, you will extend it for all issues in the project.