Security labels: What's the difference between certification and qualification?

certification-qualification-signature-electronique
Paris, 5 March 2012 - When you want to acquire or use a product that is sensitive in terms of security (such as an operating system, electronic signature tool, smart card or PKI), one of the first things you will invariably ask yourself is: how you can evaluate their security. And how can you know if this security is sufficient to meet your needs?
 
To answer these questions, it is important to be able to refer to labels issued by official bodies that offer an independent and impartial assessment of the products and services you want to use.
 
In France, the Agence Nationale de la Sécurité des Systèmes d'Information (ANSSI), the national authority in charge of the security of information systems, is responsible for organizing the issue of security labels, on behalf of the Prime Minister, to trusted products and service providers.


Currently, ANSSI recognises and issues two main types of labels. These labels are used for:
  • certifying products
  • qualifying products and services
But what's the difference between the two types? Here are some answers for people who are not security experts, together with a description of the main certifications and qualifications issued by ANSSI today.
 
 
«Certifying and qualifying doors»
 
signature-electronique-securisee
In order to help understand the difference between these two types of label, let's start with a deliberately simplistic and wholly imaginary example relating to the physical security of buildings. Let's consider what certification and qualification could mean, in IT security terms, for a door.
 
All doors do not all offer the same level of security, depending on their characteristics and in particular on the number of locking points they may have. There are doors with a single locking point, two points, three points, five points, and so on.
 
« Certifying a door » in the IT-security sense would consist of creating an official document stating the number of locking points it has (Security Target) and then having this checked by an independent body to confirm that the door does indeed have the number of locking points specified in the document. We can then imagine all ranges of doors (with one, two, three, five locking points, etc) being certified.
 
« Qualifying a door »on the other hand, would mean drawing up an official reference framework that covers only those doors having at least three locking points. The process of qualification would consist of having an official independent body check that the door in question does indeed have three or more locking points.
 
To summarize: in this example, certification consists of "certifying that the door has the number of locking points it says it has" and qualification consists of "certifying that the door has at least three locking points". As you can see, the two notions are similar but not exactly the same in terms of security.
 
 
Certifying products
 
General principle
 
The certification of a product is carried out in two stages:
 
  1. The first step consists of defining, for a particular "product scope", a set of security objectives and then associating a number of security rules and regulations to these objectives so that they can be met. These elements are gathered together in a document called a "Security Target". This document enables the evaluator to assess and position the level of security of the product and to agree on what is being certified (the chip, the reader and the chip, etc). Vendors are fairly free when it comes to drafting the Security Target and can choose to focus on whatever they like when it comes to security and product scope. Saying that a product is certified does not mean much unless the level of security and the product components involved in the certification are specified.

  2. The second step simply involves having an independent, certified laboratory confirm that the product respects the security target established. This is the assessment procedure itself.
 
Common Criteria certification
 
logo criteres communs
The main product certification issued by ANSSI is the Common Criteria certification . The Common Criteria is an internationally recognised standard for evaluating computer security (a list of the countries in which it is recognised is available here). This standard covers all phases of product development including design, architecture, the robustness of the algorithms implemented, its development environment, how it is delivered to the end user, and so on.
 
The question of security is considered a whole. The CC defines seven incremental levels of security, running from EAL1 to EAL7, with each level adding additional security constraints. To prove that a product respects the recommendations described in the standard, a certification process has been put in place. In France, this certification process is controlled by ANSSI, which appoints specialist independent laboratories called Centres for Information Technology Security Evaluation (CESTI in French) to evaluate the product and its compliance with the CC standard.
 
These laboratories must first be accredited by the French Accreditation Committee (COFRAC) according to the standard NF EN ISO/CEI 17025 and must be independent of the company that designed the product, in order to provide the customer with impartial confirmation that the product meets the criteria set out in the Security Target and in the standard.
 
The certificates issued by ANSSI on behalf of the Prime Minister confirm that the products certified conform to a technical specification called the Security Target, which is a specification defining all the security mechanisms put in place for the product. The job of the testing laboratory is to check whether the product conforms to this specification. This is the reference document that the evaluators come back to again and again throughout the evaluation process.
 
The Common Criteria does not require any specific features to be included in the target. As explained above, each vendor is free to define the content of its own Security Target provided it respects the rules defined in the standard. However, in order to evaluate certain "families" of products - for instance, electronic signature devices, smart cards, timestamping applications - based on similar criteria and on standardised features, there exist certain "common" targets called Protection Profiles that have been validated by ANSSI or one of the other international bodies (the BSI for instance, which is the equivalent of ANSSI in Germany). A list of Protection Profiles is available on the l'ANSSI website. A Protection Profile defines a set of security features that are common to each product family (for instance, for an electronic signature device, one such feature would consist of requiring authentication to be performed before the signature takes place). It is thus possible to define a Target in accordance with a given Protection Profile and this compliance is what will be evaluated during the evaluation process.
 
Once all the evaluation tasks have been performed successfully, the evaluating laboratory submits a file to ANSSI containing the results of the evaluation work carried out. Based on this file, ANSSI issues a certificate attesting that, on the day of its signature, a particular version of a product or system complies with the requirements listed in its Security Target. A certificate must be renewed over time or to keep up with the progress of the state of the art in terms of security.
 
 
Qualifying products and services
 
General principle
 
The qualification of products and services is usually carried out in two stages:
 
  1. The first step consists of choosing a security framework, the aim of which is to set the level of security to be reached in order to obtain the qualification in question. A framework is a set of security rules and best practices. Unlike certification, qualification does not allow vendors to adapt this framework - it sets out all security requirements. In other words, in a qualification process, vendors are not free to focus on whatever they like when it comes to security or product scope. They must comply with a certain number of well-specified security obligations if they are to obtain the qualification. These requirements often include obtaining a certification.

  2. The second step consists of have an accredited independent auditor check that the product respects the framework associated with the qualification. This is the evaluation procedure itself.
 
Qualifying a product
 
The qualification of security products is complementary to Common Criteria certification. Legally, in France it is the subject of Chapter II of Decree No 2010-112 of 2 February 2010 taken for the application of articles 9, 10 and 12 of Ordinance No. 20005-1516 of 8 December 2005 on electronic exchanges between administrative authorities and users. A qualification can attest to the conformity of a security product to the RGS (Référentiel Général de Sécurité, or General Security Framework). This document and its numerous appendices define a set of binding security rules for French administrations when it comes to securing their information systems.
 
The qualification process provides for three levels of qualification: basic, standard and enhanced. Again, it is ANSSI that deals with the qualification cases and issues the qualification certificates. At the standard and enhanced levels, the procedure is based on the Common Criteria, but ANSSI also performs some additional checks. ANSSI validates the Security Target for itself to ensure it meets the needs of the administrations. ANSSI also performs (or has someone else perform) tests on the cryptographic algorithms and their implementation. All of the algorithms described in the Target that provide a security service will be subject to analysis and tests (i.e. attacks) in order to confirm their robustness.
 
Qualifying a service according to the RGS framework
 
Critères Communs
As well as qualifying products, ANSSI also deals with the qualification of trusted security services. When you operate a service, it is not enough to use certified or even qualified material - you also need to operate it in the appropriate security conditions. In fact, the qualification of trusted service providers is the subject of Chapter IV of Decree No. 2010-112 of 2 February 2010 taken for the application of articles 9, 10 and 12 of Ordinance No. 20005-1516 of 8 December 2005 relating to electronic exchanges between administrative authorities and users. Such a qualification can certify that a trusted service provider complies with the rules of the RGS framework that apply to the service provider in question. The qualification process provides for three levels of security: *, ** or ***. Once the level is determined, the service provider is then audited by an accredited organisation. To date, only the LSTI has received this accreditation. The aforementioned Decree (Article 4) states that administrations should, as a general rule, always have recourse to qualified trusted service providers, and that any exceptions must be justified.
 
Qualification as it applies to electronic evidence
 
preuve-electronique
In addition, there are two other service qualification processes that are specific to the field of electronic evidence.
 
One relates to the providers of electronic certification services as defined in Decree No. 2001-272 ("Electronic Signature") and described in the Decree of 26 July 2004. This Decree defines and regulates the qualification of certification service providers capable of delivering qualified certificates for electronic signatures that can be deemed reliable. For more information, I refer you to this post (FR).
    The other relates to electronic timestamp service providers as defined in Decree No. 2011-434 ("Electronic timestamp") and described in the Decree of April 20 2011. This Decree defines and regulates the qualification of electronic timestamp service providers capable of implementing a timestamping process that can be deemed to be reliable. For more information, I refer you to this post (FR)