Sections
MVs
Qualitative Experiments
Quantitative Experiments
Settings
🛠️ QL Trueness
Overview
Other Names - Accuracy, Bias Test, Correctness Verification, Accuracy Assessment, Method Accuracy Evaluation, Bias Measurement, Correctness Validation, Accuracy Verification, Bias Analysis, Method Accuracy Check.
Trueness is one of the primary MV experiments. It involves comparing your instrument’s actual results with those an EQA qualified sample. The mechanism is similar to Method Comparison, but compares actual results with highly validated results for comparison. Few samples may be used for comparison due to stringent sample requirements.
Lab Juice
Definitions
Name | Definition |
---|---|
Actual | Actual represents the real-world value measured by the instrument being evaluated. |
Agreement | Whether two values, usually the actual and reference values acceptably agreeable, based on testing requirements and/or Error Allowance. |
Agreement Percent | The percentage (%) of paired Results within Agreement amongst the number of Samples measured. |
Agreements | The total number of measurements between actual and reference value in agreement. |
Eligible (Sample and Results) | Samples that provide the necessary information for statistical calculations to be performed. Usually this is possessing both an actual and expected result. |
EQA Sample | A specimen provided by an external quality assessment organization to participating laboratories for testing. The results from these samples are used to evaluate the laboratory's performance and accuracy by comparing it against a benchmark or other participating laboratories. |
Experiment | A sampling of results that are statistically analyzed and interpreted in the evaluation of testing performance. |
External Quality Assessment (EQA) | External Quality Assessment (EQA) evaluates a laboratory's performance by comparing its results with other labs through external sample testing. This ensures accuracy and consistency in diagnostic testing. |
False Negative (FN) | The number of false negative results where the test incorrectly missed the presence of the substrate when compared to the reference method. |
False Positive (FP) | The number of false positive results where the test incorrectly detected the presence of the substrate when compared to the reference method. |
Label | The general identification used to identify a specific Sample. |
Method Comparison (MC) Experiment | An MV experiment used to estimate the systematic difference on the basis of the differences observed between the methods.
See: |
Method Verification | A systematic process to evaluate whether the performance of a medical Test meets quality goals set by a validation. Performance evaluations may usually be found in a manufacturer insert. |
Min. Agreement | The minimum percentage of agreement required between the test results and the reference results from the acceptance criteria for the experiment to pass. |
Min. Samples | An Acceptance Criteria of the minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled. |
Neg Label | A label used in a Qualitative test to express a that a sample does not have detectable levels of substrate |
Negative | Value from Qualitative or Semi-Quantitative set in Tests that expresses non-detection of the measured substrate. |
Negative Predictive Value (NPV) | The proportion of negative test results that are true negatives. |
Pos Label | A label used in a Qualitative test to express a that a sample has detectable levels of substrate |
Positive | Values Qualitative or Semi-Quantitative set in Tests that can express a Result that indicates a Positive Result. |
Positive Predictive Value (PPV) | The probability that a Positive Result indicates the Expected analyte Result is a positive. |
Qualitative | A Result Type for a Test that can only have two possible options |
Qualitative Agreement | In Qualitative testing a matching pair of Actual and Reference Results are considered in Agreement |
Reference Method | Reference methods are well-established, highly accurate, and precise analytical procedures used as standards to evaluate the performance of other methods. |
Reference Value | The value/result that represents the true value from a trusted reference source such as a verified instrument, EQA, or a commercial product. |
Result(s) | A value or determination collected by measurement or calculation. |
Sample(s) | Individual specimens collected for testing representing the source. In MV experiments with Runs, this can refer to to the number of concentration levels used. |
Sensitivity (Qualitative) | The ability of the test to correctly identify true positives. |
Specificity (Qualitative) | The ability of the test to correctly identify true negatives. |
True Negative (TN) | The number of true negative results where the test correctly identified the absence of the substrate when compared to the reference method. |
True Positive (TP) | The number of true positive results where the test correctly detected the presence of the substrate when compared to the reference method. |
What is the Trueness Experiment?
The Trueness Experiment assesses the accuracy of a measurement method, indicating how close the average of a series of test results is to the true value of the analyte. High trueness means that the method produces results that are consistently accurate and free from systematic errors or bias. This is crucial for ensuring the reliability and validity of diagnostic tests.
The Trueness Experiment involves comparing the test results from the method being verified against known reference values or a gold standard.
External Quality Assessments (EQAs) and Interlaboratory Comparisons (Interlab) are integral to evaluating trueness. In EQAs, samples are distributed to multiple laboratories to assess their performance and ensure accuracy. Interlab comparisons involve multiple laboratories testing the same samples and comparing their results to ensure consistency and accuracy across different settings.
How to Perform Qualitative Trueness
1. Define Acceptance Criterias
In Cualia’s Trueness Experiment there are two Acceptance Criterias:
Min. Samples: The minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled.
Cualia Trueness has a limit of max 10 samples. They should be of high EQA validity.
Cualia Recommendations: Verifications: 5-10 samples Validations/LDTs: 20-30 samples
Min. Agreement (%): The minimum percent of N results where the actual measurements are in agreement with their expected values.
Generally it is recommended to set Min. Agreement to 95% because it strikes a balance between clinical significance and statistical reliability in many statistical tests such as the Bland-Altman.
2. Sample Selection
For Trueness in Method Verification (MV), sample selection is crucial and should involve using gold standard samples that have been validated and agreed upon by multiple sources, such as External Quality Assessments (EQA) and Interlaboratory Comparisons (Interlab). These samples ensure that the method’s accuracy is assessed against the highest standards, providing a reliable benchmark for trueness evaluation.
3. Testing Samples
Measure each sample on a new instrumentation (actual). Record the results in comparison with the gold standard reference values, label and source either in the Cualia app or on your own platform. Include the label and the source.
4. Use the Cualia MV App
Make sure the data is entered into the experiment with the right acceptance criterias.
Preparation Checklist
General Experiment Recommendations: Dos and Don’ts
Data Table
There are 5 relevant columns in the Qualitative Trueness Data Table:
Label | Free text - The general identification used to identify a specific Sample.
Source | Free text - The original source of the Sample such as Patient, EQA, or the name of the Manufacturer
Actual | Pos or Neg Label only - Represents the real-world value measured by the instrument being evaluated.
Reference | Pos or Neg Label only - The true value as measured from the reference method.
Result | Read Only - Indicates whether the measurements are within agreeable range.
Qualitative Trueness Results
Name | Definition |
---|---|
Min. Samples | An Acceptance Criteria of the minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled. |
Min. Agreement | The minimum percentage of agreement required between the test results and the reference results from the acceptance criteria for the experiment to pass. |
Sample(s) | Individual specimens collected for testing representing the source. In MV experiments with Runs, this can refer to to the number of concentration levels used. |
Agreement | Whether two values, usually the actual and reference values acceptably agreeable, based on testing requirements and/or Error Allowance. |
Agreements | The total number of measurements between actual and reference value in agreement. |
Agreement Percent | The percentage (%) of paired Results within Agreement amongst the number of Samples measured. |
True Positive (TP) | The number of true positive results where the test correctly detected the presence of the substrate when compared to the reference method. |
True Negative (TN) | The number of true negative results where the test correctly identified the absence of the substrate when compared to the reference method. |
False Positive (FP) | The number of false positive results where the test incorrectly detected the presence of the substrate when compared to the reference method. |
False Negative (FN) | The number of false negative results where the test incorrectly missed the presence of the substrate when compared to the reference method. |
Sensitivity (Qualitative) | The ability of the test to correctly identify true positives. |
Specificity (Qualitative) | The ability of the test to correctly identify true negatives. |
Negative Predictive Value (NPV) | The proportion of negative test results that are true negatives. |
Positive Predictive Value (PPV) | The probability that a Positive Result indicates the Expected analyte Result is a positive. |
Qualitative Trueness Acceptance
When both samples and agreement values satisfy the acceptance criterias, the experiment will be in a Passed state indicated with a green check.
Samples
Samples - The number of eligible samples tested. In the results bar, the color will turn green when the number of samples has satisfied the Min. Samples acceptance criteria to indicate it has passed.
Agreement
Agreement - The proportion (%) of test results that match the reference results. In the results bar, the color will be green and be considered passed when it satisfies the Min. Agreement or Min. Between-Day Agreement acceptance criteria.
On this page
- 🛠️ QL Trueness
- Overview
- Lab Juice
- Definitions
- What is the Trueness Experiment?
- How to Perform Qualitative Trueness
- 1. Define Acceptance Criterias
- 2. Sample Selection
- 3. Testing Samples
- 4. Use the Cualia MV App
- Preparation Checklist
- General Experiment Recommendations: Dos and Don’ts
- Data Table
- Qualitative Trueness Results
- Qualitative Trueness Acceptance
- Samples
- Agreement