EDPB CEF DPO report - a deep dive

Highlighted file:

Please note: I haven't documented this to 'take down' the EDPB or anyone involved, to point fingers or come across as a know-it-all.

I'm doing it because firstly, I hope that they take the feedback onboard to improve on future Coordinated Enforcement Framework (CEF) action, and secondly, they should reassess using these results for further national initiatives.

Especially when they state that the results provide "valuable global results that can be considered significant". I disagree. And I hope the EDPB doesn't rely too heavily on this when updating the DPO guidelines, either.

Honestly, I think we should spend the most energy on key takeaways and what we can implement in practice. And the plan was to start this article with these...

But when writing about years of experience in the industry/field (question 11), I again noticed something odd, starting with the Dutch results: they only add up to 35,8%! Others are more than 100%. 😳 So, as I sadly conclude at the end of this article, I'll just leave it at this.

Background

In March 2023, the EDPB launched their second Coordinated Enforcement Framework (CEF) action: Designation and position of DPOs.

The CEF, the EDPB writes:

is a structure for DPAs to work together, coordinate or facilitate enforcement actions on a pre-defined topic and with an agreed-upon methodology.

Note that last point there for my later comments.

With the CEF, the EDPB hope to promote compliance, empower data subjects to exercise their rights, raise awareness, and increase the knowledge of DPAs.

On 17 January, the EDPB published a press release, with links to the final report, including the DPAs national reports and the statistics.

First, if you look at the press release from 2023, the EDPB says the questionnaire would be sent to DPOs.

But then, in January, they write "Various organisations, as well as DPOs were contacted"... In fact, only 7 DPAs directed their survey exclusively at DPOs.

🚨 A nuance with massive implications for the credibility of the survey.

Second, despite initially announcing that 26 DPAs would participate, the 🇷🇴 Romanian results are missing from the final report without any explanation. However, I've had it confirmed that DPOs received and returned a survey to the DPA. Still wonder what happened there.

Next, the EDPB participated in a LinkedIn live with IAPP, which I attended. Unfortunately, I didn't get an answer to my questions about the methodology...

Since then, I've posted several LinkedIn posts where I gradually uncovered more concerns, as detailed below.

And on 29 February, we published our Grumpy GDPR episode, extending an invitation to the EDPB for a follow-up discussion. Let's see if they accept. I really hope they do.

Rie's findings and point of view

If you've followed my previous posts and listened to the podcasts episodes on Serious Privacy and Grumpy GDPR, you're familiar with my perspective on this issue. Below, I've outlined my main findings and criticisms.

The pressing questions now are: Does it really matter? Is this a problem? If so, how? And is there any value in this for us as DPOs?

In my view, yes, it definitely matters. And it's a serious problem.

First of all, I'm not just grumpy; I'm concerned. It's astonishing to see so many organisations and individuals dedicate effort and resources to a coordinated project, which ends up so deeply flawed (taxpayer money 💸).

It's been a massive undertaking. 26 DPAs, the EDPB, almost 43k organisations and 19k DPOs contacted, "more than 17,000 replies received and analysed", press releases, website updates. Just think of the time spent planning, executing, in meetings etc.

Second, the EDPB base their recommendations on these findings and encourages organisations, DPAs and DPOs to take the "valuable insights" into account to tackle the identified challenges.

But a flawed methodology leads to flawed results. How can you write a credible report with this backdrop? How can anyone conclude on the status quo of the GDPR in their country with flawed results, let alone create guidance based on it?

Consider my findings below and let me know what you think. Am I overly grumpy?

Concerning or puzzling national examples

Let's start with what triggered my concern to begin with, question 22:

This was not aligned with my experience, nor with what I've heard from other DPOs for years. Notice that the title is incorrect, because responses were provided by organisations, DPOs or both.

🇦🇹 Then Austria caught my attention, with the very aligned answers of all 11 respondents, which also are remarkably positive throughout:

Even more concerning, the Head of the Austrian DPA, Matthias Schmidl, said in their CEF press release that "The GDPR has arrived in practice" with "positive results" for their audit of the financial sector.

Based on a survey of 11 organisations where not one DPO responded.

Unsurprisingly, then, "The investigation of the Austrian banking sector did not identify any significant problems with regard to the role of DPOs."

And, worse: "This is not decided yet, but since no significant problems could be identified concerning the DPOs of the banking sector, no major actions by the Austrian Supervisory Authority will probably be necessary in this regard."

But this is not only a national issue. The Austrian results affect the EDPB report too, boosting especially the max values with as much as 71%, but impacting averages, minimums and medians as well.

Back to my concern of the applicability of the results. Remember, both the EDPB and DPAs intend to use the results to address the challenges identified. And here are other alarming issues:

🇩🇪 The Bavarian DPA BayLDA intends to "make our conclusions usable for all of the more than 36,000 data protection officers communicated to us". Let that sink in: thirty-six thousand DPOs.

How many would you say they'd have to ask, to get a solid enough basis to make such conclusions usable for all of them?

Well, fasten your seatbelt because this will shock you: they contacted 36 organisations. Yes, you read that right: thirty-six. Of which 35 responded, of which 14 were DPOs. 😳 How they conclude that these results can lead to true insights for THIRTY-SIX thousand DPOs is beyond me.

🇪🇪 Next up is Estonia, sharing that they selected organisations "from both the public and private sectors ... rural municipality and city governments, ministries, banks, hospitals and other institutions" and continue to say that "such a diverse sample gives us a good overview and also raises the value of the results".

How many do you think they surveyed, in all those sectors, types of organisations - this diverse sample? 19. With 16 (all DPOs) responding.

🇫🇷 What about the French CNIL, with their 30,000 DPOs across 80,000 organisations (as of March 2022)? They surveyed 15 public bodies and got 14 responses (no DPOs).

This isn't a problem per se, but it becomes one when a DPA, based on the results, 1) makes general conclusions for the status quo of the GDPR in their country, and/or 2) determines follow-up actions (like creating new guidance).

🇳🇱 Finally, I have to mention the astonishingly low response rate of the Dutch DPA: 7.46%. When I asked on LinkedIn and with Curated DPO newsletter subscribers, DPOs told me the survey was confusing with several non-sensical questions. In addition, respondents could be identified, deterring people from responding. More on that below.

🇧🇪 The response rate for the Belgian DPA is also very low, at 13.75%, but reasons are unknown. If you have any insights, please share!

I can go on. I have several other examples of clear errors and inconsistencies in the data, like responses not adding up to 100% where it should.

I also reviewed every conclusion the EDPB makes in their press release. Some seem to be only inferred as I couldn't find any matching questions, for example for necessary skills and knowledge to do their work and receive regular training. Others are imprecise; consulted in most cases aren't backed by all the data.

Then, several questions are confusing or even contradicting. For example, asking about mere experience and expert knowledge in the same question (Q9) makes no sense. And how can an answer option be "part-time" to a question directed at DPOs working on a fulltime basis (Q19)? 🤔

❌ And this is where the problems started, with the questionnaire.

As I mentioned initially, the DPAs were meant to use an agreed-upon methodology. They started out with one, creating the survey on the EDPB level. Then it went sideways from there, as national DPAs were permitted to tailor the survey and recipients, ref. the EDPB's own comment:

SAs who sent the commonly-built questionnaire sometimes added more questions or possible answers to the common questionnaire or sometimes removed or slightly modified certain questions.

(And Poland used their own questionnaire.)

Why didn't the EDPB create one single questionnaire that every DPA had to use, and had to send to only DPOs? I just don't get it.

Because of the many deviations, the results are incomparable. Just consider the section on methodology on pages 9-11:

🚨 However, my primary concern overall is this: several DPOs have told me that they didn't respond truthfully to the survey, in fear of being audited later by the DPA.

As many of us feared last year, the CEF was likely doomed to fail from the start with 'investigations' in the mix: DPAs will implement the CEF in a number of ways:

• to aid fact-finding exercise or questionnaires
• to identify if a formal investigation is warranted;
• commencement or follow-up of a formal investigation;

Now we know.

Key takeaways..?

Back to the question: Is there any value in this for us as DPOs?

Initially, I thought there were some interesting findings. And as I said at the very beginning, the plan was to write more about these. But I'm too unsure about the results to dare make any conclusions.

That said, the EDPB's list of recommendations isn't unsound, either. We need more resources, training, and clear tasks, to become more independent, to be in direct contact with top management, and to report regularly.

There are, however, two upcoming DPO surveys, so let's hope we use those chances wisely. Hint hint!

🔗 LinkedIn posts: #1 initially super excited, #2 Q22 perks my interest (and concern), #3 🚨 shocking discovery