Overview

1. Classification Output
2. Validation Positives
3. Validation Negatives

Installation

1. frm_Acc_Assess.frm
2. frm_Acc_Assess.frx
3. Read_me.html (this document)

The Interface

Calculate Button
Before Accuracy Assessor can operate, an ArcMap Project must exist containing the classification, validation present, and validation absent layers. All these layers must be vector polygons. Once loaded, begin the script by clicking the 'Calculate'. Accuracy Assessor will first check to see if the classification Layer has more than one record. If it does, it will ask the user if they would like to dissolve the classification now. Clicking Yes will call a routine that will dissolve the geometry of this layer (using GP.Dissolve), clicking No will cancel the Accuracy Assessor script. Once the features are dissolved, a new shapefile is added to the map. If geometry is dissolved, the user will have to click 'Claculate' a second time to complete the script. Accuracy Assessor has the ability to accommodate a geometric buffer around the classification product, which may be necessary to adjust for spatial mis-registration. The buffer generates a temporary shapefile and saves to your hard drive, and removes these shapefiles after processing is complete. If you do not want to use a buffer, leave this value at '0', which is the default (nothing is written to your hard drive).

Print Button
Pressing the Print Form button will print the form, in its current state, to your default printer. A warning box will appear asking you to confirm if you want to print to avoid accidental waste of paper.

Export Button
Pressing this button will export an ASCII comma delimitated file to a user defined directory. If the same file is written to multiple times, various assessments will be appended to each other, to make simple importing into spreadsheet or other software for display and comparison.

Clear Button
This button clears the form and variables.

Known Limitations

1. Accuracy Assessor expects and requires only polygon features to operate.
2. Accuracy Assessor currently requires singular (dissolved) geometry for the input classification.
3. Accuracy Assessor is not currently able to overwrite (replace) existing files on export or save.
4. Accuracy Assessor currently does not produce metadata for generated shapefiles.

References

Congalton, R.G. & Green, K. (1999). Assessing the accuracy of remotely sensed data: Principles and practices,(137) Boca Raton, Florida: Lewis Publishers

Congalton, R.G. & Mead, R.A. (1983). A quantitative method to test for consistency and correctness in photointerpretation. Photogrammetric Engineering & Remote Sensing, 49(1), 69-74

Foody, G.M. (2004). Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy. Photogrammetric Engineering & Remote Sensing, 70(5), 627-633

Foody, G.M. (2002). Status of land cover classification accuracy assessment. Remote Sensing of Environment, 80185-201

Contributing Scripts
This script modified some open source examples to derive the final product. These scripts include but may not be limited to:

Technical Notes

Kappa
In this script, Kappa is calculated in the standard fashion, using methods described by Congalton (1991). Mathematically, this relationship is described as:

Where Po is the value from Row1 Column1 and Pc is the expected to return if the classification were due to random chance. In this study, we assumed that a random classification would return approximately 50% of positives, which corresponds one-half of the total of Column 1. As such, the script calculates the following relationship :
kappa = Round((Val(PP) - (Val(C1) / 2)) / (1 - Val(C1 / 2)), 2)

Z Stat
The Z Statistic in this script is calculated using the McNemar's test, which is useful for tabulating the significance of a 2x2 matrix (Foody, 2004). Mathematically, this relationship is described as:

Where f represents the frequency values of confusion matrix elements i and j (Foody, 2004). Computationally, this script calculates the zstat value as follows:
zstat = Round((Val(AP) - Val(PA)) / ((Val(AP) + Val(PA)) ^ 0.5), 2)

Contact Info