DSMM Assessment: Six Stages, Evidence and Rework

Bottom line first: the hard part of a DSMM assessment is not the policy documents, it is whether the six stages have real technical controls and operating records. A policy that states "data is encrypted", "data is masked", "auditing is in place" is only a statement; at assessment you have to be able to open the system and see the encryption configuration, export the masking task records, and pull an audit log with a timestamp. If it cannot be produced, the item does not count — however well written, it scores zero.

DSMM is a data security capability maturity model, sorting capability into 5 levels: 1 informal execution, 2 planned and tracked, 3 well defined, 4 quantitatively controlled, 5 continuously optimising. The object of observation is a matrix: across it lie the six stages of the data lifecycle — collection, transmission, storage, processing, exchange and destruction; down it lie the four capability dimensions — organisation, policy, technology and personnel. Six stages by four dimensions, and every cell has to carry "what was done" and "what proves it".

On process, a self-assessment or gap analysis usually comes first, to establish the distance between the current state and the target, and then a qualified assessment body carries out the formal assessment. What the assessment examines is evidence: policy documents, process records, technical controls (classification and grading, masking, encryption, auditing), and actual operating data.

This article follows the order "set the level first → what each of the six stages is examined for → how to close the four dimensions → how to run the self-assessment → what evidence the formal assessment examines → common rework points → how to sequence it against DCMM / ISO 27001 and regulatory requirements". No standard numbers are given, and no fee or timeline figures — both depend on the assessment scope, the size of the company and the arrangements of the body, and are subject to the body carrying out the assessment.

1. Set the level first: among the five levels, many companies stall between Level 2 and Level 3

On the level, start with where the requirement comes from: which level the tender document names, what a regulatory inspection referred to, which tier the group's or the customer's data security requirement sits at. If the text states it, prepare to the text, subject to the tender document or the written requirement of the regulator — do not guess from industry habit.

Why stall between 2 and 3? Because the two levels are different in nature. Level 2 proves "someone is watching"; Level 3 proves "there are definitions, processes and records". From 1 to 2, more people, more meetings and more checks can push it up; from 2 to 3, the actions have to be written into policy, embedded in systems, and actually run for several rounds leaving traces behind. The technology column is where this gap opens widest — with no classification and grading labels, no actual masking and encryption configuration, and no centrally searchable audit logs, better policy text and more training cannot fill it in.

Two reminders. First, higher is not automatically better: Level 4 calls for quantitative indicators and an alerting mechanism, and forcing it without several years of operating data gives a poor return. Second, no body should promise in advance which level you will reach. The level is judged independently by the assessment body under the assessment rules, and this article makes no commitment about the outcome either.

2. Collection and transmission: where the data comes from and how it travels

Of the six stages, collection is the source and transmission is the pipe. If the source is unclear, the grading, masking and auditing that follow have nowhere to land — if you do not know what data has come in, you cannot label it.

What the collection stage is asked about:

• Is there a data source register? Business systems, third-party partners, public channels, endpoint imports — which sources exist in each category, who approved them, and what fields came in. A register that does not match the systems throws everything after it off.
• Is there authorisation and notice before collection? For collection involving personal information, the form of notice and consent and the way it is retained have to be producible. The specific requirements are subject to the regulatory requirements in force.
• Are classification and grading labels applied at collection? This is a key action of the technology dimension: is data newly coming in automatically or semi-automatically classified and graded when it is stored or connected, or is it patched afterwards once something has gone wrong.
• Are the interfaces controlled? For externally provided collection interfaces, SDKs and the tracking point inventory, is there authentication and a field whitelist, or can anyone call them.

What the transmission stage is asked about: whether the channels are encrypted (service-to-service calls, cross-network transmission, mobile-side reporting), whether cross-network and cross-border transmission has separate approval, and whether third-party interfaces have a register and expiry review. Cross-border is a point of high regulatory attention; whether it is permitted and what formalities are required are subject to the regulatory requirements in force — do not judge from experience.

3. Storage and processing: encryption, masking and permissions — check configuration, not claims

This is the part where companies feel comfortable in self-assessment and get pressed hardest on site. The reason is simple: the technical controls in storage and processing all have "looks like it exists" substitutes — a licence for encryption software, the audit function built into the database switched on, one round of manual masking. All of them can be written up in a briefing as "capability in place", and none of them is a capability that runs sustainably.

Storage is examined on four points:

1. Which layer the encryption reaches. Disk or storage volume encryption, or database table column-level encryption? The two guard against different risks. The policy says "sensitive fields are encrypted" while only disk encryption is in place — one question on site makes that gap clear.
2. How keys are managed. Who holds the keys, how often they are rotated, whether they are stored separately. Whether encryption is done well often turns on key management, and this is the step usually skipped altogether.
3. Where and how long it is kept. Data partitioning and storage location, retention period and the rule for handling expiry. The retention period needs a basis; "keep it forever" is not one.
4. Backup and restore. Whether backups are encrypted, whether a restore drill has been run, and where the drill record is. A backup policy without a drill record means it has never been verified.

Processing is examined on four points: whether permissions are reduced to the necessary scope (granted by role and scenario, not packed by department), whether development, testing and analytics scenarios use masked data, whether operations are audited, and whether accounts form a closed loop from creation to revocation. On masking specifically, the test is whether there are rules, whether there are versions, and whether there are task records — hand-editing a few rows of data and exporting them to test does not count as a technical control.

4. Exchange and destruction: the two ends that are easily missed

Exchange and destruction are the two stages of the six with the lowest profile, and the two where gaps cluster. The reason is not hard to see: collection, storage and processing run every day and someone is watching them; providing data externally is an occasional action, and destruction comes round once every few years, so both easily stop at the level of "we know it has to be done".

The exchange stage: the scenarios in which data is provided externally have to be listed in full — customer delivery, partner sharing, regulatory reporting, intra-group access. Four questions for each scenario: who approved it, which fields were approved, how the recipient uses it, and when it is recalled or terminated. On the technical side, look at whether interface pushes have a field whitelist, whether file exports carry a watermark and an audit trail, and whether shared data is masked. The form of evidence is a sharing register + approval records + interface call logs; only when the three line up is the loop closed. A single line in an email saying "agreed to provide" is not a register.

The destruction stage: how expired data is deleted, how media are handled, how accounts are closed, and whether copies in backups are handled at the same time. The typical hole here is "the primary database was deleted, the backups and the endpoints were forgotten" — ask on site "is that copy still in the backup?" and the answer is often not a good one. Destruction needs approval and execution records; for important media, two-person witnessing and a disposal certificate are advisable. The destruction record has to answer: what data, when, by whom, in what way, and who verified it.

5. Closing the four capability dimensions: organisation, policy, technology, personnel

The four dimensions are not four sets of documents; they are four faces of the same thing. Treating them as four parallel preparation tracks is a common misunderstanding at the self-assessment stage — the result is one set of policies, another set of technical work and a third set of training, none of which references the others, and one question on site pulls them apart.

Of the four dimensions, technology is the watershed, because it cannot be filled in with words. Organisation can issue a document, policy can be revised, personnel can be trained — all of that can be done internally; technical controls need systems, configuration and operating data, and cannot be staged. This is also why many companies write "largely conformant" against all four dimensions in self-assessment and then lose points concentrated on the technology items in the formal assessment.

The recommended order of closing is: settle the organisation and the owners first → then revise the policy so that it describes "what is actually being done" → embed the technical actions into system steps in parallel → use training and assessment to fix people's actions. Do it the other way round — write the policy first and then hunt for technology to match it — and the policy becomes a wish list that cannot be delivered.

6. Running the self-assessment / gap analysis: work back from evidence, not from clause scoring

The value of a self-assessment is finding gaps early, not issuing yourself a pass. In method, walking through the data flow is more effective than scoring clause by clause.

1. Draw the data flow. For each category of important data: where it comes in (collection), which channels it passes through (transmission), which databases and files it lands in (storage), who uses it and how (processing), who it has been given to (exchange), and how it is handled at expiry (destruction). Once it is drawn, it turns out that the data flow of many systems and the description in the policy are not the same line.
2. Lay out the matrix. Six stages × four dimensions, twenty-four cells. Fill three sentences into each cell: is there a written rule? Is there a matching action in the system? Can a record be exported? If all three are "yes", add one more: where is the evidence, and who can produce it on the spot.
3. Take evidence on site. Do not ask in a meeting "have we done this"; have people open the system: sample one customer record and look at its label, export yesterday's audit log, find the execution record of one masking task. The "yes" obtained in a meeting and the "yes" that can be opened and seen on the spot are often not the same thing.
4. Produce the gap list. Classify by nature, because different types are closed in completely different ways — that is the real output of the self-assessment.

One signal in a self-assessment report deserves caution: all twenty-four cells "conformant", not one real gap. Such a self-assessment collapses at the first sample on site — not because the capability is absent, but because the self-assessment failed to do its job of finding problems. The other way round, a self-assessment that finds twenty or so specific gaps, each with a way to close it, an owner and a target state, is a useful document.

Also, the gap analysis criteria have to be aligned with the assessment body. On the same requirement, different assessors may look from different angles; asking about the judgement criteria, the sampling approach and the form of evidence at the self-assessment stage saves far more effort than explaining on site. The specific requirements are subject to the current assessment rules of the body carrying out the assessment.

7. What evidence the formal assessment examines: documents, records, systems, plus an interview

The formal assessment usually runs in two parts: a document review first, to see whether the material is complete and consistent with itself; then an on-site stage of interviews, sampling and system checks. The passing standard differs across the four categories of evidence.

One part of the on-site stage deserves separate preparation: the demonstration. The assessor may ask an engineer to operate on the spot — look up the classification label of one data record, run a masking task, search audit logs by data object, export a sharing register. If only one person can do these, or they depend on remote support from the vendor, the risk is high. The method is plain: write the actions to be demonstrated into a script and have the person who actually does the work rehearse them a few times, rather than having the project manager explain on their behalf.

Interviews have their own rules. The assessment asks the people who really do the work: who set this data's classification and on what basis; how many alerts came up last quarter and how they were handled; who signed when data was provided externally. Questions of that kind cannot be passed by memorising the material. The interviewees should be people who can explain "why it was decided this way", not people who speak well.

On independence: consulting and assessment are two roles, and the party that helps you build the system should not also issue the assessment conclusion. With claims of the form "we assess it ourselves and we know where the result will land", ask one question first: which body issues the conclusion, and where can its qualification be checked. The list of assessment bodies and the scope of their qualification are subject to the information published by the competent authority for the current period; this article lists no names.

8. Common rework points, and how to sequence it with DCMM and ISO 27001

Rework is usually not a question of capability but of the form of the evidence — the work was done, but nothing checkable was left behind. The following come up often and are worth a self-check before you start.

• The policy says "encrypted", and no configuration can be produced on site. If any one of the encryption scope, the key management and the rotation records cannot be explained, the item is left hanging.
• Classification and grading stop at the header row. There is a grading method and a grading list, but the fields in the system carry no label, and a sampled record cannot be traced to a level.
• Masking depends on manual work. No rule library and no task records; data in the test environment is "whoever needs it edits it".
• The permission matrix is written by department. Not by role and system; in actual operation super accounts are shared, so the matrix and reality are two different things.
• Logs are switched on but cannot be exported. Retained locally, short cycle, not collected centrally; at sampling, no record for the specified period can be produced.
• Sharing actions have no register. External provision rests on email and verbal agreement; the field scope, the recipient and the recall date cannot be stated.
• Destruction only covered the primary database. Copies in backups, endpoints and test environments were not handled, and media scrapping has no disposal record.
• Dates that run backwards. A policy is released later than the execution records it governs, or a record carries a version number that has not yet taken effect. This kind of hard defect is very hard to explain.
• The self-assessment passes on every item. Not one real gap, and the first sample in the formal assessment collapses.
• Everything handed to an outside party. The consultant knows the system better than the employees, who cannot answer "who defined this rule, and how often does it run".

Now the combination. These three often appear in the same tender document for a data-related project, but they examine different things and do not substitute for one another.

In practice there are three points of experience:

• A company that has done ISO 27001 starts DSMM more lightly, but the two cannot be converted directly. ISO 27001 is an organisation-level ISMS; the skeleton of risk assessment, permission management, auditing and internal audit can be reused directly. DSMM is centred on the data lifecycle and needs the technical evidence laid out again stage by stage. The inference "having ISO 27001 means data security is in place" does not hold.
• The crossing point of DSMM and DCMM is data security. Classification and grading, access control and audit trails overlap, so one base set of policy documents can be shared; but the evidence criteria have to be prepared separately — DCMM looks at where data security sits within the data management system, DSMM looks at the maturity of the security capability itself. Sharing a base document saves duplicated labour, not duplicated evidence gathering.
• Sequence by working back from the deadline, not by "how complete the system looks". Whichever is an admission item in the tender document goes first; when both are admission items, the usual route is to put the ISO 27001 skeleton in place first (policy, document control, internal audit, corrective action) and then build up to the stage depth of DSMM, which reuses more.

As for the relationship with regulatory requirements, keep the two things apart. The common drivers are the compliance obligations around data security and personal information protection law, regulatory inspections, and the qualification thresholds of government and enterprise tenders. Those are the practical push behind "why do this now". But DSMM is a capability assessment, not a compliance conclusion — it states which level your data security capability has reached and cannot replace a compliance judgement; conversely, the problems found in a compliance inspection are often a ready-made gap list for the DSMM technology dimension, and the two can be closed against each other. Whether you are compliant, and which obligations you have to meet, are subject to the regulatory requirements in force.

For a conceptual overview, see the DSMM knowledge base page; the full DCMM process is set out in this article; on choosing between the systems, see the ITSS / CMMI / DCMM / CSMM comparison; on the information security side, see ISO 27001 and ISO 20000, and the overall map is on the management systems overview.

Action checklist

Go through this in order before you start, and move on only when every line has a concrete answer:

• ☐ I can state clearly where the target level comes from — tender document, regulatory requirement or customer admission, and what the text actually says
• ☐ Which business lines, systems and data categories this assessment covers has been drawn out, and the scope can be stated
• ☐ Each of the six stages has its own contact person, rather than everything resting on one information security person
• ☐ We have drawn one complete data flow, continuous from collection through to destruction
• ☐ Classification and grading have landed as labels in the system, and one sampled record can be traced to its level
• ☐ Encryption, masking and auditing can be opened and shown on the spot — configuration and logs, not only screenshots and contracts
• ☐ External sharing has a register, destruction has approval and disposal records, and copies in backups are covered too
• ☐ We have run a round of self-assessment or gap analysis and can produce twenty or so specific gaps
• ☐ We have people who can demonstrate on site: engineers who can look up labels, export logs and run masking
• ☐ I am clear on the sequence of DSMM against DCMM and ISO 27001, and on which material can be shared and which has to be evidenced again

Ticked fewer than five of these ten, and now is not the time to book an assessment — configuring the technical controls and letting the records run is far faster than booking the on-site days and gathering evidence afterwards. Where things are unclear, you can also contact us directly and go through the six stages one by one.

Conclusion: get the evidence running first, then talk about the level

What a DSMM assessment tests is a very plain thing: whether what you say, what runs in the system and what the records keep are the same thing. The policy shows how it should be done, the configuration shows what actually protects, the logs show it really was protected that way, and the interviews show that the people doing it know why. String the four together and the capability holds; miss one and you have to go back to the system on site to fill it in, not back to the document to edit it.

So the order is: set the level and scope first → lay out the six stages × four dimensions matrix along the data flow → run the self-assessment to find the gaps → close organisation and policy, configure the technical controls → let the mechanism really run for a period and let the records grow → only then discuss the assessment schedule. Do it the reverse way, and however neat the material is, it will not survive the round of demonstrations and sampling on site.

▌Not sure which of the six stages has the widest gap? Leave a comment or message me with "DSMM self-check" and I will send you a checklist broken down by stage.