Concepts and Terms
FAE and AInspector for Firefox use the OpenA11y Evaluation Library the following concepts are important in understanding the terms and relatiohnships of the rules, rulesets and evaluation results of the library. The OpenA11y Evaluation Library is based on the requirements
|Rule||A low-level accessibility compliance requirement with numerous components, the most basic of which are a definition, an evaluation function, and result messages. When a rule is evaluated, it is assigned a rule result value.|
|Ruleset||A set of rules that embodies specific standards compliance goals. Each rule within a ruleset is designated as either required or recommended, according to its relationship to the compliance goals of the ruleset.|
|Evaluation||The application of all evaluation functions of all rules in the selected ruleset to their corresponding target resources in a web page, and the subsequent collection of all evaluation results and messages, at both the rule and element levels, into a dataset.|
|Rule result||The aggregate result of a rule evaluation with respect to its target resources (all of its target elements and/or the entire page).|
|Element result||The result of a rule evaluation with respect to one of its target elements.|
Rule and element result values
|Violation||V||One or more target elements fail the requirements of a required rule|
|Warning||W||One or more target elements fail the requirements of a recommended rule|
|Manual Check||MC||Unable to programmatically determine a result - evaluation requires human inspection and knowledge of the associated WCAG 2.0 requirement.|
|Pass||P||All rule requirements were satisfied|
|Hidden||H||Target elements were not evaluated because they are hidden (either visually or from assistive technologies)|
|Not Applicable||N/A or ‘-’||Rule was not applied because there were no relevant target elements|
More about rules, target resources and result values
- A rule tests target resources for specific accessibility features.
- Some rules are fully automated and return pass or fail results and messages.
- Other rules can only identify the need for human judgement to determine if an accessibility requirement has been met. These return manual check results and messages.
- If target resources are not found on the page, the rule result is designated as not applicable.
- If target resources for a rule are hidden, the rule result is designated as hidden. The hidden resources are included in the element results list to enable developers to consider their accessibility if and when they become available to users.
- Rules also include information on accessibility techniques and links to related resources to help developers understand how to meet the requirements of a rule.
Implementation Score Definitions
- The implementation score is based on the rule results across all pages and ranges from 0 to 100.
- '-' means there were no rule results for that group (e.g. rules were not applicable).
- The following messages give a general idea of the overall implementation of the rules for a website.
|C||100||Complete||Complete means all rules have passed. There were either no manual checks or all manual checks have been resolved.|
|R||0||Required Manual Checks||
|na||-||Not Applicable||No markup was found that identifed a known or possible accessibility issue.|
Note: Future versions of FAE will allow you to mark manual checks as, pass, fail or not applicable.
Implementation Score Computation
Implementation Score = P / (P + F + MC)
Each variable represents the number of element level results for a particular rule, where:
- P = elements that passed
- F = elements that failed
- MC = elements that require a manual check
The score ranges in value from 0 to 100, that is from no implementation to all accessibility requirements met. In the future, FAE will allow you to change manual checks to pass, fail or not applicable, and will recompute the Implementation Score based on the updated manual checking results.
Note: When a manual check is changed to not applicable that element result is removed from the implementation score computation.
Implementation Score Combinations
Implementation scores are rule-based and are computed for the following combinations:
|1||1||Element level results determine the implementation score for a rule on a specific page.|
|1||All||The implementation score for a page is the average of all implementation scores for all rules on the page, with all rules being equally weighted.|
|All||1||The implementation score for a rule across all the pages in a website is based on the aggregation of all element results for the rule across all pages.|
|All||All||The implementation score for a website is the average of the website rule implementation scores.|
Example Rule Implementation Scores on Individual Page
|Description||Pass||Fail||Manual Check||Implementation Score|
|A rule with scope of element, like "Image must have
|A rule with scope of page like "Page must have
|A rule with scope of element with elment results of pass, fail and manual checks that may occur for the color contrast rule.||134||29||37||67|
Example Rule Implementation Score for All Pages in a Website
This is a hypothetical example for a website
|Page results for Rule X||Pass||Fail||Manual Check||Implementation Score|
|Page 1 element results for rule X.||23||12||4||59|
|Page 2 element results for rule X.||12||4||0||75|
|Page 3 element results for rule X.||45||0||0||100|
|Website element results for Rule X||80||16||4||80|