Sections
MVs
Qualitative Experiments
Quantitative Experiments
Settings
Qualitative Method Comparison (MC)
Overview
Other Names - Comparison of Methods, Comparative Method Analysis, Intermethod Comparison, Cross-Method Validation, Method Equivalence Study, Analytical Method Comparison, Benchmark Method Comparison, Performance Comparison Study, Parallel Method Testing, Method Correlation Study, Alternative Method Evaluation
Method Comparison is one of the primary MV experiments. It involves comparing your instrument’s actual results with those of a true reference instrument. This verifies that the new method produces results that are in agreement
Lab Juice
Definitions
Name | Definition |
---|---|
Result(s) | A value or determination collected by measurement or calculation. |
Agreements | The total number of measurements between actual and reference value in agreement. |
Specificity (Qualitative) | The ability of the test to correctly identify true negatives. |
Sensitivity (Qualitative) | The ability of the test to correctly identify true positives. |
Negative Predictive Value (NPV) | The proportion of negative test results that are true negatives. |
False Positive (FP) | The number of false positive results where the test incorrectly detected the presence of the substrate when compared to the reference method. |
False Negative (FN) | The number of false negative results where the test incorrectly missed the presence of the substrate when compared to the reference method. |
True Negative (TN) | The number of true negative results where the test correctly identified the absence of the substrate when compared to the reference method. |
True Positive (TP) | The number of true positive results where the test correctly detected the presence of the substrate when compared to the reference method. |
Reference Method | Reference methods are well-established, highly accurate, and precise analytical procedures used as standards to evaluate the performance of other methods. |
Negative | Value from Qualitative or Semi-Quantitative set in Tests that expresses non-detection of the measured substrate. |
Min. Samples | An Acceptance Criteria of the minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled. |
Eligible (Sample and Results) | Samples that provide the necessary information for statistical calculations to be performed. Usually this is possessing both an actual and expected result. |
Method Comparison (MC) Experiment | An MV experiment used to estimate the systematic difference on the basis of the differences observed between the methods.
See: |
Pos Label | A label used in a Qualitative test to express a that a sample has detectable levels of substrate |
Neg Label | A label used in a Qualitative test to express a that a sample does not have detectable levels of substrate |
Method Verification | A systematic process to evaluate whether the performance of a medical Test meets quality goals set by a validation. Performance evaluations may usually be found in a manufacturer insert. |
Experiment | A sampling of results that are statistically analyzed and interpreted in the evaluation of testing performance. |
Reference Value | The value/result that represents the true value from a trusted reference source such as a verified instrument, EQA, or a commercial product. |
Spiked | A prepared sample, often using a Blank that has been "spiked" with a measurable analyte. |
MV | A broad term that can be used synonymously with Performance Evaluation, Method Validation and/or Method Verification. |
Sample Source | The original source of the Sample such as Sample, EQA, or the name of the Manufacturer. |
Qualitative | A Result Type for a Test that can only have two possible options |
Qualitative Agreement | In Qualitative testing a matching pair of Actual and Reference Results are considered in Agreement |
Label | The general identification used to identify a specific Sample. |
Method Validation | A systematic process to evaluate whether the performance of a medical Test meets quality goals to be used for medical testing. |
Actual | Actual represents the real-world value measured by the instrument being evaluated. |
Sample(s) | Individual specimens collected for testing representing the source. In MV experiments with Runs, this can refer to to the number of concentration levels used. |
Control | A Quality Control material, usually developed by a manufacturer, with a known Expected Result used for monitoring Test performance. |
Laboratory Developed Test (LDT) | A method developed by the laboratory or a regulator approved/validated method that has been modified. |
Positive | Values Qualitative or Semi-Quantitative set in Tests that can express a Result that indicates a Positive Result. |
Agreement | Whether two values, usually the actual and reference values acceptably agreeable, based on testing requirements and/or Error Allowance. |
Agreement Percent | The percentage (%) of paired Results within Agreement amongst the number of Samples measured. See: Agreements |
Positive Predictive Value (PPV) | The probability that a Positive Result indicates the Expected analyte Result is a positive. |
What is Method Comparison?
Method Comparison in medical laboratory verification is a process where a new method of testing is compared with a standard or established method. This is done to ensure that the new method is reliable, accurate, and consistent with the existing method. The new method should yield results that are clinically equivalent to the established method.
During this process, a variety of samples are tested using both methods and the results are compared. If the new method's results are within acceptable limits of the established method, the new method can be considered for implementation.
Method Comparison can be performed on Qualitative and Quantitative tests.
How to Perform Qualitative Method Comparison
1. Define Acceptance Criterias
In Cualia’s Method Comparison Experiment there are two Acceptance Criterias:
Min. Samples: The minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled.
Cualia Recommendations: Verifications: Quantitative 20-30 samples. Qualitative 10-30 samples. Validations/LDTs: 80-100 samples
Min. Agreement (%): The minimum percent of N results where the actual measurements are in agreement with their expected values.
Generally it is recommended to set Min. Agreement to 95% because it strikes a balance between clinical significance and statistical reliability in many statistical tests such as the Bland-Altman.
Sample Selection
Choose a representative set samples that reflect the diversity and variability expected in routine testing. Ensure samples cover the full range of analyte concentrations and conditions relevant to the test.
It is recommended that most of the samples come from the true population. Some manufacturer products such as commercial kits and controls may be used. These may often be necessary for particularly difficult to obtain concentrations and conditions.
Testing Samples
Measure each sample on a reference method (expected) and on your new instrumentation (actual). Record the results, label and source either in the Cualia app or on your own platform. Include the label and the source.
Use the Cualia MV App
Make sure the data is entered into the experiment with the right acceptance criterias.
Reference Methods
Reference Methods are well-established, highly accurate, and precise analytical procedures used as standards to evaluate the performance of other methods.
For testing you will need to select a Reference Method to act as your reference result to match with the actual results of the new test.
It is recommended to use a validated and verified source of the same model/make as the analyzer being tested to ensure compatibility of the measurements.
Preparation Checklist
General Experiment Recommendations: Dos and Don’ts
Data Table
There are 5 relevant columns in the Qualitative Method Comparison Data Table:
Label | Free text - The general identification used to identify a specific Sample.
Source | Free text - The original source of the Sample such as Patient, EQA, or the name of the Manufacturer
Actual | Pos or Neg Label only - Represents the real-world value measured by the instrument being evaluated.
Reference | Pos or Neg Label only - The true value as measured from the reference method.
Result | Read Only - Indicates whether the measurements are within agreeable range.
Qualitative Method Comparison Results
Name | Definition |
---|---|
Min. Samples | An Acceptance Criteria of the minimum number of eligible samples required in order for the evaluation’s requirements to be fulfilled. |
Min. Agreement | The minimum percentage of agreement required between the test results and the reference results from the acceptance criteria for the experiment to pass. |
Sample(s) | Individual specimens collected for testing representing the source. In MV experiments with Runs, this can refer to to the number of concentration levels used. |
Agreement | Whether two values, usually the actual and reference values acceptably agreeable, based on testing requirements and/or Error Allowance. |
Agreements | The total number of measurements between actual and reference value in agreement. |
Agreement Percent | The percentage (%) of paired Results within Agreement amongst the number of Samples measured. See: Agreements |
True Positive (TP) | The number of true positive results where the test correctly detected the presence of the substrate when compared to the reference method. |
True Negative (TN) | The number of true negative results where the test correctly identified the absence of the substrate when compared to the reference method. |
False Positive (FP) | The number of false positive results where the test incorrectly detected the presence of the substrate when compared to the reference method. |
False Negative (FN) | The number of false negative results where the test incorrectly missed the presence of the substrate when compared to the reference method. |
Sensitivity (Qualitative) | The ability of the test to correctly identify true positives. |
Specificity (Qualitative) | The ability of the test to correctly identify true negatives. |
Negative Predictive Value (NPV) | The proportion of negative test results that are true negatives. |
Positive Predictive Value (PPV) | The probability that a Positive Result indicates the Expected analyte Result is a positive. |
Qualitative Method Comparison Acceptance
When both samples and agreement satisfy the acceptance criterias, the experiment will be in a Passed state indicated with a green check.
Samples
Samples - The number of eligible samples tested. In the results bar, the color will turn green when the number of samples has satisfied the Min. Samples acceptance criteria to indicate it has passed.
Agreement
Agreement - The proportion (%) of test results that match the reference results. In the results bar, the color will be green and be considered passed when it satisfies the Min. Agreement or Min. Between-Day Agreement acceptance criteria.
On this page
- Qualitative Method Comparison (MC)
- Overview
- Lab Juice
- Definitions
- What is Method Comparison?
- How to Perform Qualitative Method Comparison
- 1. Define Acceptance Criterias
- Sample Selection
- Testing Samples
- Use the Cualia MV App
- Reference Methods
- Preparation Checklist
- General Experiment Recommendations: Dos and Don’ts
- Data Table
- Qualitative Method Comparison Results
- Qualitative Method Comparison Acceptance
- Samples
- Agreement