The Crown Estate

Weekly Report Methodology

Back to home

1. Introduction

This document sets out the methodology used to produce the Weekly Wales Devolution Activity Analysis Report. The weekly report is the foundational intelligence product in the monitoring framework, providing a structured, near-real-time picture of the online discourse surrounding Welsh devolution, independence, and related constitutional questions. It feeds directly into the Monthly Review Report, which synthesises weekly outputs into a higher-level strategic assessment.

The weekly report is designed as a multi-layered narrative intelligence pipeline, combining structured open-source data collection with human validation and strategic synthesis. It moves from broad ecosystem scanning (national and local media, political announcements, institutional statements) to targeted actor tracking, myth monitoring, quantitative social data analysis, and executive-level signal extraction for The Crown Estate (TCE).

The methodology transforms raw weekly activity into a structured, board-ready intelligence output. It is built around qualitative insight rather than raw metrics alone, although quantitative data (engagement rates, follower movements, view counts) is captured and analysed as supporting evidence for narrative judgements.

2. Pipeline Overview

The execution framework operates as a sequential pipeline of eleven steps, each building on the outputs of the previous stages. The pipeline can be grouped into four functional phases:

  • Phase 1 - Collection: News monitoring and target account tracking across pro-devolution, anti-devolution, and neutral ecosystems (Steps 1-2).
  • Phase 2 - Validation: Date-range enforcement and duplicate removal to produce a clean, verified content set (Steps 3-4).
  • Phase 3 - Analysis: Executive summary, intensity indexing, TCE signal extraction, recurring myth assessment, and topic frequency mapping (Steps 5-9).
  • Phase 4 - Social Data: Platform-level engagement metrics, insight generation, and Top 10 most active accounts identification (Steps 10-11).

Certain steps involve direct human review and editorial judgement. These are identified in the relevant sections below.

3. Phase 1: Collection

3.1 Step 1: News Monitoring

Purpose

Step 1 establishes the factual baseline of the week by harvesting news coverage from national and local sources. It captures both pro-devolution, anti-devolution, and neutral developments, ensuring the pipeline begins with a comprehensive view of the information environment rather than a partisan slice.

Process

AI research agents are tasked with searching the web for real articles published during the reporting week. The search scope covers Welsh and UK-wide media, government publications, political party communications, think-tank outputs, and academic commentary. The agent is instructed to find articles from all sides of the devolution debate, including neutral coverage that does not take an explicit pro or anti position.

Deliverables

The step produces three outputs for each side (pro, anti, and neutral):

  • Executive Summary: Two to three professional paragraphs summarising the key themes and developments of the week.
  • Executive Bullets: Six to eight concise, information-rich bullets detailing specific developments, each linked to the relevant source.
  • Content Table: A structured table with columns for Content Title (a summary, not just a headline), Account/Who, Date, Platform, Link (with numeric reference), and Topic (a short thematic label).
  • References: A full list of sources with numeric citations used throughout the summary and bullets.

3.2 Step 2: Target Account Monitoring

Purpose

Step 2 narrows the lens from the broad media landscape to a predefined list of tracked accounts and organisations. This ensures that narrative momentum is captured not only through institutions but through the individuals and groups actively shaping devolution discourse. The account lists are maintained in a separate database.

Process

The AI research agents search the web for devolution-related news published during the reporting week that features or mentions any account on the tracked lists. The search is conducted separately for pro-devolution, anti-devolution, and neutral account groups.

Deliverables

The step produces three distinct output blocks:

  • Pro-Devolution Output: Executive bullets summarising key pro-devolution stories connected to tracked accounts, plus a content table populated from the pro-devolution account list.
  • Anti-Devolution Output: Executive bullets summarising key anti-devolution stories connected to tracked accounts, plus a content table populated from the anti-devolution account list.
  • Neutral Output: Executive bullets summarising key neutral stories connected to tracked accounts, plus a content table populated from the neutral account list.

Each content table follows the same structure as Step 1 (Content Title, Account/Who, Date, Platform, Link, Topic/Notes).

4. Phase 2: Validation

4.1 Step 3: Date Range Check

Before analysis begins, all collected content items are checked against the reporting week's date range. Any items that fall outside the target week are flagged and removed. This step ensures temporal integrity across the analysis pipeline.

4.2 Step 4: Duplicate Check

The four content tables produced by Steps 1 and 2 (News, Pro-Devolution News, Anti-Devolution News, Neutral News) are cross-checked for duplicates based on the Link column. Where the same article or source appears in more than one table, duplicates are flagged and consolidated to avoid double-counting in downstream analysis. The output is a single, deduplicated content set that forms the basis for all subsequent analytical steps.

5. Phase 3: Analysis

5.1 Step 5: Executive Summary

Purpose

The Executive Summary distils the full week's collection into a concise, slide-ready narrative overview. It is the single output most likely to be read by senior stakeholders and must therefore balance comprehensiveness with brevity.

Process

A large language model (LLM) is provided with the full outputs from Steps 1 and 2 and tasked with producing a synthesis. The model identifies the dominant themes, key actors, notable patterns, and any splits between institutional and media coverage. The summary is typically four to seven sentences and avoids bullet points in favour of connected prose.

Output

A single narrative paragraph or short set of paragraphs capturing the essential character of the week's devolution activity. The summary identifies who drove the conversation, which platforms and outlets were most active, and what the overall trajectory of the discourse was (escalation, de-escalation, consolidation, or fragmentation).

5.2 Step 6: Intensity Index

Purpose

The Intensity Index provides a simple, comparable measure of the balance between pro-devolution, anti-devolution, and neutral narratives for the week. It allows week-on-week tracking of which side of the debate held greater agenda-setting power, and how much of the discourse sat outside the explicit pro/anti framing.

Process

A large language model (LLM) is provided with all collected content from Steps 1 and 2 and instructed to calculate a comparative intensity split across pro-devolution, anti-devolution, and neutral content. The calculation considers both the volume of content items on each side and the qualitative salience of those items, weighing factors such as whether a narrative was driven by official government announcements, viral social media moments, or routine campaign activity.

Output

A percentage split across the three categories (e.g. Pro 55% / Anti 30% / Neutral 15%), accompanied by a one-sentence slide-ready insight explaining the balance.

5.3 Step 7: Signals for The Crown Estate

Purpose

This step extracts the strategic implications of the week's activity specifically for The Crown Estate (TCE). It translates general devolution discourse into actionable intelligence relevant to TCE's licence to operate in Wales, stakeholder relationships, and reputational positioning.

Process

A large language model reviews all collected content and extracts four to six strategic bullets focused on:

  • TCE's licence to operate in Wales,
  • implications of the May Senedd elections,
  • political risks,
  • narrative shifts,
  • escalation or de-escalation in rhetoric,
  • and stakeholder positioning.

The model draws connections between the week's events and TCE's specific exposure.

Output

Four to six strategic bullets, each framed as an actionable insight for TCE decision-makers. These bullets are designed to be directly usable in board-level briefings.

5.4 Step 8: Recurring Myths Tracker

Purpose

The Recurring Myths Tracker monitors a fixed set of ten myths and misconceptions about The Crown Estate that circulate in Welsh public discourse. Its function is to provide early warning when a myth is spiking in prominence, enabling TCE to prepare proactive or reactive communications.

The Ten Myths

The following myths are assessed each week:

  • Myth 1: Profits from Wales go to the King / Royal Family.
  • Myth 2: Devolution will instantly keep 100% of 'Welsh' CE money in Wales and fix council budgets.
  • Myth 3: TCE charges people to use beaches / puts paywalls on access.
  • Myth 4: No money is reinvested in Wales; everything is siphoned to London.
  • Myth 5: TCE is a private royal company, secretive and unaccountable.
  • Myth 6: Offshore wind is all profit for TCE / Monarchy; Wales won't see jobs or supply-chain benefits.
  • Myth 7: TCE harms nature and fisheries in Welsh waters.
  • Myth 8: TCE sets energy bills and profits when prices rise.
  • Myth 9: TCE blocks community energy and local ownership in Wales.
  • Myth 10: TCE owns and runs Welsh castles like Caernarfon / Harlech.

Process

An AI research agent searches for news and social media discussions during the reporting week to identify which myths are currently spiking. The search is directed specifically at these ten myths, and the agent is instructed to assess each one against the available evidence.

Output

For each myth: a brief summary of its current status, an evidence strength rating (High, Medium, or Low), and a reason explaining why it is or is not spiking. The myths are then ranked by prominence. The output also includes a references table linking all cited content items.

5.5 Step 9: Topic Frequency

Purpose

The Topic Frequency analysis provides a quantitative breakdown of the themes discussed during the week. It serves as both a summary tool and a charting input for visual reporting.

Process

Topics are drawn from the Topic column of the content tables produced by Steps 1, 2, and 8. An AI agent groups and cleans the topics, merging duplicates and closely related themes, and calculates the frequency of each.

Output

A frequency table (Topic | Frequency) for charting, allowing the week's thematic composition to be visualised alongside previous weeks.

6. Phase 4: Social Media Data

6.1 Step 10: Platform Data Monitoring

Purpose

Step 10 captures quantitative social media performance data across Facebook, Instagram, TikTok, and YouTube for all tracked accounts. This data provides the engagement evidence base that supports and contextualises the qualitative narratives identified in earlier steps.

Data Collection

Data is collected directly via third-party social media analytics services. For each tracked account, the following metrics are captured where available: engagement rate, follower or subscriber change, media or upload count change, view change, likes change, and talking-about change. Metrics vary by platform; for example, Facebook surfaces likes and talking-about counts, while YouTube provides subscriber and view changes.

Insight Generation

Once collected, the raw platform data is passed to a large language model with a prompt to produce three to five clear, concise, slide-ready bulleted insights per platform. The insights focus on patterns such as: which accounts saw the largest engagement spikes, where posting volume and engagement diverged, which accounts gained or lost followers, and any notable cross-platform patterns.

Output

Per-platform insight bullets (Facebook, Instagram, TikTok, YouTube), plus the underlying structured data (account-level records with all captured metrics) stored in the report's output.

6.2 Step 10b: X (Twitter) Data Monitoring

Purpose

X (formerly Twitter) data collection operates alongside the platform monitoring but follows a different methodology due to the platform's data access constraints.

Current Approach: Estimation Model

At the time of writing, X data is collected via an estimation model rather than a direct data pull. An AI research agent collects available public data points for each tracked account (current follower count, following count, total tweets, and tweets published during the reporting week) and calculates derived metrics.

The key estimated metrics are:

  • Corrected Engagement Rate: Calculated as (Total Engagements / (Followers x Tweets)) x 100, which accounts for both audience size and content volume and produces realistic percentages typically ranging from 0.1% to 20%.
  • Estimated Follower Change: Derived from a conversion model where Base Formula = Total Engagements / Conversion Rate x Activity Multiplier, with the conversion rate adjusted by engagement rate (high engagement above 5% reduces the conversion rate by 50%; medium engagement between 1% and 5% reduces it by 25%) and the activity multiplier scaled between 1.0 and 1.2 based on tweet frequency.

Estimation Anchoring

The estimates are anchored to known data points from publicly available sources and scaled by account type (party accounts, individual politicians, grassroots campaigns, niche organisations), political prominence, and the week's activity intensity. Welsh political grassroots accounts typically see 3-5% engagement due to smaller but passionate audiences, national figures tend toward lower rates due to massive follower bases, and universities and institutional accounts sit between 0.5% and 1.2%.

Planned Evolution

The X data collection methodology is designated as 'still in progress' and is intended to transition to a direct data pipeline comparable to the other platforms. Until that transition is complete, X metrics should be interpreted as modelled estimates rather than observed values, and this distinction is noted in the report outputs.

6.3 Step 11: Top 10 Most Active This Week

Purpose

The Top 10 identifies the accounts that dominated the week's social media activity across all monitored platforms, providing a quick-reference snapshot for stakeholders.

Process

A large language model compiles all available social media performance data from Instagram, YouTube, TikTok, Facebook, and X for the reporting week. It analyses all accounts across platforms and determines overall activity level using a blended assessment of engagement rate, volume of uploads or posts, and follower or subscriber growth. The model then identifies the ten most active accounts and, for each, determines which platform they were most active on.

Output

A ranked list of the ten most active accounts. The output also includes a one-sentence insight summarising where overall activity concentrated during the week.

7. Human Touch and Quality Assurance

While the pipeline relies heavily on AI agents for collection and first-pass analysis, several steps involve direct human review and editorial judgement. These include:

  • Review and validation of AI-collected news items for accuracy, relevance, and completeness.
  • Editorial oversight of executive summaries and TCE signals to ensure strategic framing is appropriate and actionable.
  • Quality checks on myth assessments to guard against false positives (a myth rated as spiking without genuine supporting evidence) or false negatives (a genuine spike missed by the AI agent).
  • Sense-checking of intensity index splits against the analyst's own reading of the week.
  • Verification that social media data aligns with observed activity and that estimated X metrics carry appropriate caveats.

The combination of AI speed and human judgement is central to the methodology's design. AI handles breadth and structure; human analysts provide depth, context, and accountability.

8. Data Sources and Platforms

The weekly report draws on the following data sources:

  • News and media: Welsh and UK national media, GOV.WALES, Senedd.wales, party websites, think-tank publications, academic journals, and international outlets covering Welsh affairs.
  • Social media platforms: X (formerly Twitter), Facebook, Instagram, TikTok, and YouTube.
  • Data APIs: Third-party social media analytics services for Facebook, Instagram, TikTok, and YouTube metrics.
  • Tracked account lists: Maintained in a database, covering both pro-devolution, anti-devolution, and neutral actors across political parties, grassroots campaigns, media outlets, academics, think-tanks, environmental organisations, and individual public figures.

9. Limitations and Caveats

The following limitations apply to the weekly report and should be considered when interpreting its outputs:

  • AI collection scope: AI research agents search the open web and may not capture content behind paywalls, in private groups, or in Welsh-language sources that are underrepresented in search indices.
  • X (Twitter) estimation: Until a direct data pipeline is established, all X metrics are modelled estimates. They are directionally useful but should not be treated as precise measurements.
  • Intensity Index subjectivity: The pro/anti/neutral split involves qualitative judgement by a language model and is not a statistical measurement. It is designed to track directional shifts rather than precise ratios.
  • Myth assessment inference: In some weeks, a myth may be assessed as spiking based on contextual triggers (e.g. a government announcement that creates conditions for a myth to resurface) rather than direct observation of the myth being repeated. The evidence strength rating (High/Medium/Low) is designed to communicate this distinction.

10. Relationship to the Monthly Review

The weekly report is the primary input to the Monthly Review Report. At the end of each reporting month, all weekly reports are reviewed in their entirety and synthesised into three strategic sections: the Narrative Timeline, the Persona and Algorithm Analysis, and the Influencer Network and Evidence Board. The weekly report's content items, platform metrics, myths data, and intensity indices all feed into the monthly synthesis process. The methodology for the Monthly Review is documented separately.