# 10. Data Ingestion


Data Ingestion shows how data reaches Fyscal ARCX, so administrators can check that each import ran and what it loaded. **Batch Ingestion** lists the scheduled import jobs for Customers, Accounts and Transactions files, lets you download source and error rows, and holds their schedule. **Watchlist Ingestion** lists the watchlist files loaded from LSEG and from the Internal Watchlist, and is where you upload your own Internal Watchlist file.

<dl class="chapter-facts"><div><dt>For</dt><dd>Administrators</dd></div><div><dt>In the menu</dt><dd>Fyscal ARCX → Data Ingestion</dd></div><div><dt>Reading time</dt><dd>About 14 min</dd></div></dl>

::: checks
- [Batch Ingestion](#s-10-1): check each import job's records, skipped rows and status.
- [Filter batch jobs](#s-10-2): find jobs by date, file type or status.
- [Investigate skipped rows](#s-10-7): find jobs that did not load every row and download the rows that were skipped.
- [Change the ingestion schedule](#s-10-3): set a new frequency and up to three run times a day.
- [Watchlist Ingestion](#s-10-4): check which watchlist files were loaded and whether they succeeded.
- [Search, filter and sort watchlist files](#s-10-5): find a file by name, status or source.
- [Upload an Internal Watchlist file](#s-10-6): add subjects to, or remove them from, the Internal Watchlist with a CSV file.
:::

The **Data Ingestion** menu under **Fyscal ARCX** has two items: **Batch Ingestion** and **Watchlist Ingestion**. Transaction data comes in through Batch Ingestion as the **Transactions** file type.

::: note AML context
Screening and monitoring can only cover the data that was loaded. A skipped customer row is a customer who may not be screened, and a skipped transaction row is a transaction no rule sees. Checking ingestion results is part of the control, not housekeeping.
:::

## Batch Ingestion {#s-10-1}

::path[Fyscal ARCX → Data Ingestion → Batch Ingestion]

<dl class="key-facts"><div><dt>File types</dt><dd>Customers, Accounts, Transactions</dd></div><div><dt>One job per</dt><dd>File type and run, with a Job ID</dd></div><div><dt>Downloads</dt><dd>Source CSV (Total Records) and error rows CSV (Skipped Rows)</dd></div><div><dt>Schedule</dt><dd>Up to 3 run times a day, set with Change Schedule</dd></div></dl>

Check Batch Ingestion to confirm that customer, account and transaction files were imported, and to find the rows that were skipped. Batch Ingestion shows the ingestion schedule and a history of ingestion jobs. Each job imports one file type (Customers, Accounts or Transactions) and shows how many records it read, ingested and skipped.

<figure class="diagram flow"><ol class="dg-steps"><li class="dg-step"><span class="dg-title">Source file</span><span class="dg-sub">Customers, Accounts or Transactions</span></li><li class="dg-step"><span class="dg-title">Ingestion job</span><span class="dg-sub">One job per file type, with a Job ID</span></li><li class="dg-step"><span class="dg-title">Validation</span><span class="dg-sub">Job statuses include Validating and Validated</span></li><li class="dg-step"><span class="dg-title">Ingested or skipped</span><span class="dg-sub">Each row is counted in Ingested or Skipped Rows</span></li><li class="dg-step is-key"><span class="dg-title">Error rows CSV</span><span class="dg-sub">Downloaded from the Skipped Rows number</span></li></ol><figcaption><span class="cap-num">Diagram 10.1</span> — How a batch file is ingested</figcaption></figure>

Diagram 10.1 follows one file through Batch Ingestion. A job reads the source file (**Total Records**), passes through validation, and loads the rows it can (**Ingested**). The rest are counted in **Skipped Rows**, and that number downloads them as an error rows CSV. The job's final status is shown in **Status**. See [Investigate skipped rows](#s-10-7).

::figure[Batch Ingestion: the schedule bar (1), Resume and Change Schedule (2), the filters (3), Refresh (4), the job list (5) and a linked count (6).]{src="data-ingestion/01-batch-ingestion-overview.png" id="fig-data-ingestion-batch-ingestion-overview"}

::: legend
1. **Schedule bar.** Shows the schedule state (**Inactive** in the example), **Current Frequency** (**Daily at 09:00**), **Next Run** and **Last Run**. When there is no value, the screen shows a dash (-).
2. **Resume** and **Change Schedule**. **Resume** is shown next to **Change Schedule** while the schedule is **Inactive**. **Change Schedule** opens the **Change Ingestion Schedule** dialog (see [Change the ingestion schedule](#s-10-3)).
3. **Filters.** **Job Date** (**Pick a date range**), **File Type** (**All File Types**) and **Status** (**All Status**). See [Filter batch jobs](#s-10-2).
4. **Refresh** reloads the job list.
5. **Job list.** One row per job, newest first. Below the list, **Showing 1-20 of 113** gives the range and total, and **Page 1 of 6** has first, previous, next and last page buttons.
6. **Linked counts.** Underlined numbers in **Total Records** and **Skipped Rows** are download links (see the column table below).
:::

| Column | What it shows | Format or example |
| --- | --- | --- |
| **Job Date / ID** | When the job ran, with the job number below it after **JOB ID:**. A copy icon sits next to the job number. | 14 Oct, 03:41:31 PM<br>JOB ID: 268 |
| **File Type** | The data the job imported, as a coloured tag. | **Customers**, **Accounts**, **Transactions** |
| **Total Records** | Records in the source file. When the value is above 0 it is an underlined blue link that downloads the source CSV. | 1,037 |
| **Ingested** | Records loaded. Plain text. | 1,000 |
| **Skipped Rows** | Rows not loaded. When the value is above 0 it is an underlined red link that downloads the error rows CSV. | 34 |
| **Status** | The job's status tag (see the status table below). | Completed |
| **Duration** | How long the job ran. A dash (-) is shown for the job in **Validating** status. | 4s, 8m, 2m, - |

::figure[Two job rows. In job 237 the Total Records value 100,037 (1) and the Skipped Rows value 34 (2) are links; job 238 found no files and has no links.]{src="data-ingestion/05-batch-job-row.png" id="fig-data-ingestion-batch-job-row"}

::: note
Clicking a job row does not open a detail view. The only links in a row are the underlined **Total Records** and **Skipped Rows** numbers.
:::

In most jobs, **Ingested** plus **Skipped Rows** equals **Total Records**: job 171 (Customers) shows 1,000 = 600 + 400, and job 173 (Accounts) shows 1,500 = 130 + 1,370. In some jobs the two counts add up to 3 less than Total Records. Job 237 (Transactions) shows 100,037 records, 100,000 ingested and 34 skipped (100,034). Earlier jobs 181, 136, 137 and 139 show the same gap of 3.

**Job statuses**

The **Status** filter lists ten job statuses, in this order. The screen does not describe them. Where a status appears in the example job list, the table says what that row shows.

| Status | As seen in the job list |
| --- | --- |
| **Init** | Not seen in the example list. |
| **Validating** | Job 138 (Transactions): 0 records and a Duration of -. |
| **Validated** | Not seen in the example list. |
| **Published** | Not seen in the example list. |
| **Ingesting** | Not seen in the example list. |
| **Completed** | Most jobs. Can still have skipped rows, for example job 173 (Accounts): 1,500 records, 130 ingested, 1,370 skipped. |
| **Partial Success** | Not seen in the example list. |
| **Failed** | Jobs 166 and 167 (Customers): 0 records, 0s. |
| **No Valid Record** | Not seen in the example list. |
| **No Files Found** | Jobs 269 (Accounts), 270 (Transactions) and 238 (Customers): 0 records, 0s. |

::: important
A **Completed** job can still have skipped rows. Job 173 is Completed with 1,370 of its 1,500 rows skipped. Check **Skipped Rows**, not only the status.
:::

## Filter batch jobs {#s-10-2}

::path[Fyscal ARCX → Data Ingestion → Batch Ingestion]

Use the three filters above the job list to narrow it by date, file type and status.

::: step Filter by Job Date
Click **Pick a date range** (1) under **Job Date**. A two-month calendar opens, with arrows (2) to move back and forward a month. Click the first day, then the last day of the range.

::figure[The Job Date field (1) and its calendar, showing October and November 2026 with the month arrows (2).]{src="data-ingestion/04-batch-job-date-filter.png" id="fig-data-ingestion-batch-job-date-filter"}
:::

::: step Filter by File Type
Click **All File Types** (1) under **File Type**. The list has a search box (2) and three options (3): **Customers**, **Accounts** and **Transactions**. Click one.

::figure[The File Type filter (1), its search box (2) and the options Customers, Accounts and Transactions (3).]{src="data-ingestion/02-batch-file-type-filter.png" id="fig-data-ingestion-batch-file-type-filter"}
:::

::: step Filter by Status
Click **All Status** (1) under **Status**. The list has a search box and the ten statuses (2): **Init**, **Validating**, **Validated**, **Published**, **Ingesting**, **Completed**, **Partial Success**, **Failed**, **No Valid Record** and **No Files Found**. Scroll the list to see **No Files Found**. Click a status.

::figure[The Status filter (1): the statuses from Init to No Valid Record (2). No Files Found is further down the list.]{src="data-ingestion/03-batch-status-filter.png" id="fig-data-ingestion-batch-status-filter"}
:::

::: step Reload the list
Click **Refresh** to reload the job list.
:::

## Investigate skipped rows {#s-10-7}

::path[Fyscal ARCX → Data Ingestion → Batch Ingestion]

<dl class="key-facts"><div><dt>Look for</dt><dd>A red, underlined <strong>Skipped Rows</strong> number</dd></div><div><dt>Download</dt><dd>The job's error rows CSV</dd></div><div><dt>Compare with</dt><dd>The source CSV, from <strong>Total Records</strong></dd></div></dl>

Use these steps when a job did not load every row. Skipped rows are records that were not loaded, so the customers, accounts or transactions in them are missing from the platform until they are loaded.

::: step Find jobs with skipped rows
On **Batch Ingestion**, narrow the list with **Job Date** and **File Type** if you need to (see [Filter batch jobs](#s-10-2)). Look down the **Skipped Rows** column. A value above 0 is a red, underlined number, for example 34 on job 237 in [](#fig-data-ingestion-batch-job-row).
:::

::: step Download the error rows
Click the **Skipped Rows** number. The job's error rows CSV downloads. The number does not say that it is a download, and clicking the row itself does nothing.
:::

::: step Download the source file if you need it
Click the blue **Total Records** number to download the source CSV, so that you can compare the skipped rows with the file that was sent.
:::

::: step Have the data corrected
Pass the error rows to the team that produces the file, so that the records can be corrected and sent again.
:::

::: tip
Missing data also shows up downstream. A Transaction Monitoring run with **Records Scanned** 0 had no transaction data to read; check Batch Ingestion for that period (see [Schedule Details](#s-8-4)).
:::

## Change the ingestion schedule {#s-10-3}

::path[Fyscal ARCX → Data Ingestion → Batch Ingestion → Change Schedule]

Change the schedule when batch ingestion needs to run at a different frequency or at different times. **Change Schedule** opens the **Change Ingestion Schedule** dialog. Its subtitle reads: "Select the new frequency and run times, then review before confirming."

::figure[Change Ingestion Schedule: current frequency (1), New Frequency (2), Runtimes (3), the time field and Add time (4), Next Run Preview (5), and Cancel and Review Changes (6).]{src="data-ingestion/06-change-ingestion-schedule.png" id="fig-data-ingestion-change-ingestion-schedule"}

::: legend
1. **Current Frequency**, with its time zone: **Daily at 09:00 (UTC)** in the example. Below it the dialog states: "Any schedule change — whether frequency, time, or timezone — takes effect after the next scheduled run on Fri, Oct 16, 2026, 09:00."
2. **New Frequency**: a drop-down, set to **Daily** in the example.
3. **Runtimes - up to 3 per day**: each run time is a chip (for example **09:00**) with an × to remove it.
4. A time field (**--:-- --**) and **+ Add time**, to add another run time.
5. **Next Run Preview**: the next run date and time (**Fri, Oct 16, 2026, 09:00**) and a summary, for example "Runs every day at 09:00 UTC."
6. **Cancel** closes the dialog. **Review Changes** moves on to review the change before confirming it.
:::

::: step Open the dialog
On **Batch Ingestion**, click **Change Schedule**.
:::

::: step Choose the frequency
Select a value in **New Frequency**.
:::

::: step Set the run times
To add a run time, enter it in the time field and click **+ Add time**. To remove one, click the × on its chip. You can have up to 3 run times per day.
:::

::: step Check the preview and continue
Check **Next Run Preview**, then click **Review Changes**. To leave without changing the schedule, click **Cancel** or the × at the top right.
:::

::: note
As the dialog states, a schedule change takes effect after the next scheduled run, not straight away.
:::

## Watchlist Ingestion {#s-10-4}

::path[Fyscal ARCX → Data Ingestion → Watchlist Ingestion]

Check Watchlist Ingestion to confirm that watchlist files were loaded and to see any errors. Watchlist Ingestion lists each watchlist file loaded into the platform, with its source, status, start and end times and the number of rows processed. LSEG files appear in pairs named *watchlist\_delta\_\_…* and *watchlist\_deleted\_\_…*. Files you upload yourself appear with the source **Internal Watchlist** and names beginning *internal\_watchlist\_delta\_\_* or *internal\_watchlist\_deleted\_\_*.

<figure class="diagram flow"><ol class="dg-steps"><li class="dg-step"><span class="dg-title">Watchlist files</span><span class="dg-sub">LSEG delta and deleted files, or your Internal Watchlist upload</span></li><li class="dg-step is-key"><span class="dg-title">Watchlist Ingestion</span><span class="dg-sub">Status, Total Rows Processed and Error Message per file</span></li><li class="dg-step"><span class="dg-title">Watchlist</span><span class="dg-sub">Searchable in Watchlist Search, Source LSEG or Internal Watchlist</span></li><li class="dg-step"><span class="dg-title">Screening runs</span><span class="dg-sub">Batch and Delta Screening record a Watchlist Count</span></li></ol><figcaption><span class="cap-num">Diagram 10.2</span> — How watchlist data reaches screening</figcaption></figure>

Diagram 10.2 shows the watchlist side. LSEG files and Internal Watchlist uploads are listed here, one row per file. Their records can be looked up in [Watchlist Search](#s-7-1), where the **Source** filter offers **LSEG** and **Internal Watchlist**, and each [Batch Screening](#s-7-4) or [Delta Screening](#s-7-5) run shows the **Watchlist Count** it recorded.

::figure[Watchlist Ingestion: search (1), Status (2) and Source (3) filters, Refresh (4), Upload Internal Watchlist (5), sortable columns (6), Error Message (7) and paging (8).]{src="data-ingestion/07-watchlist-ingestion-overview.png" id="fig-data-ingestion-watchlist-ingestion-overview"}

::: legend
1. **Search by File Name**.
2. **Status** filter (**All Status**).
3. **Source** filter (**All Sources**).
4. **Refresh** reloads the list.
5. **+ Upload Internal Watchlist** opens the upload dialog (see [Upload an Internal Watchlist file](#s-10-6)).
6. **Sortable columns.** **Status**, **Start Time**, **End Time** and **Total Rows Processed** have a sort icon (↑↓).
7. **Error Message** shows the error for a file, or a dash (-) when there is none.
8. Paging: **Showing 1-20 of 100** gives the range and total, and **Page 1 of 6** has first, previous, next and last page buttons.
:::

| Column | What it shows | Sortable | Example |
| --- | --- | --- | --- |
| **File Name** | Name of the ingested file. Long names are cut short with "…". | No | watchlist\_delta\_\_20261006\_051519.csv |
| **Source** | Where the file came from. | No | LSEG, Internal Watchlist |
| **Status** | Status tag of the ingestion. | Yes | Success |
| **Start Time** | When ingestion of the file started. | Yes | 06 Oct, 11:30:37 AM |
| **End Time** | When ingestion of the file ended. | Yes | 06 Oct, 11:36:48 AM |
| **Total Rows Processed** | Rows processed from the file. | Yes | 7487 |
| **Error Message** | Error reported for the file; a dash (-) when there is none. | No | - |

| Status | As seen on screen |
| --- | --- |
| **Success** | Shown on every file in the example list. |
| **In Progress** | Listed in the Status filter; not seen in the example list. |
| **Failed** | Listed in the Status filter; not seen in the example list. |

## Search, filter and sort watchlist files {#s-10-5}

::path[Fyscal ARCX → Data Ingestion → Watchlist Ingestion]

Use these controls to find a particular watchlist file, or to list only the files of one status or source.

::: step Search by file name
Type part of a file name in **Search by File Name**.
:::

::: step Filter by status
Click **All Status** (1) and choose **All Status**, **Success**, **In Progress** or **Failed** (2). A tick marks the current choice.

::figure[The Status filter (1): All Status, Success, In Progress and Failed (2).]{src="data-ingestion/08-watchlist-status-filter.png" id="fig-data-ingestion-watchlist-status-filter"}
:::

::: step Filter by source
Click **All Sources** (1) and choose (2) **All Sources**, **Internal Watchlist** or **LSEG**.

::figure[The Source filter (1): All Sources, Internal Watchlist and LSEG (2).]{src="data-ingestion/09-watchlist-source-filter.png" id="fig-data-ingestion-watchlist-source-filter"}
:::

::: step Sort the list
Click a sortable column heading: **Status**, **Start Time**, **End Time** or **Total Rows Processed**. The ↑↓ icon on that column changes to an arrow showing the sort direction. In the example below, the list is sorted by **Total Rows Processed**, lowest first.

::figure[The list sorted by Total Rows Processed: the ascending arrow on the column heading (1) and the lowest values first (2).]{src="data-ingestion/10-watchlist-sorted.png" id="fig-data-ingestion-watchlist-sorted"}
:::

## Upload an Internal Watchlist file {#s-10-6}

::path[Fyscal ARCX → Data Ingestion → Watchlist Ingestion → Upload Internal Watchlist]

Use this dialog when you need to add subjects to, or remove them from, your own Internal Watchlist. The **Upload Internal Watchlist** dialog does this using a CSV file. Its subtitle reads: "Add new subjects to the Internal Watchlist, or remove existing ones".

::figure[Upload Internal Watchlist: Add or Delete (1), the sample file (2), the upload area (3), and Cancel and Upload & Add (4).]{src="data-ingestion/11-upload-internal-watchlist.png" id="fig-data-ingestion-upload-internal-watchlist"}

::: legend
1. **Add to Internal Watchlist** / **Delete from Internal Watchlist**: chooses whether the file adds or removes subjects. **Add to Internal Watchlist** is selected when the dialog opens.
2. The sample file **internal\_watchlist\_add\_sample.csv** and **Download Sample**.
3. The upload area: "Click or drag file to this area to upload", "Only CSV format is supported", and **Browse File**.
4. **Cancel** and **Upload & Add**. **Upload & Add** is greyed out until a file is chosen.
:::

| Field or control | What it expects | Format |
| --- | --- | --- |
| **Add to Internal Watchlist** / **Delete from Internal Watchlist** | Choose the action for the file. | Toggle; Add is selected by default. |
| Instructions (Add) | "Upload a CSV of customer records to add to the Internal Watchlist (Blacklist or Whitelist). Each row needs a UID, entity details, and a rationale. Review it against the sample file before uploading." | Text |
| **Download Sample** | Downloads the sample file **internal\_watchlist\_add\_sample.csv**. | CSV |
| Upload area / **Browse File** | The file to upload. Drag it onto the area, click the area, or click **Browse File**. | "Only CSV format is supported" |
| **Upload & Add** | Uploads the chosen file. | Greyed out until a file is chosen. |

::: step Open the dialog
On **Watchlist Ingestion**, click **+ Upload Internal Watchlist**.
:::

::: step Choose add or delete
Leave **Add to Internal Watchlist** selected to add subjects, or click **Delete from Internal Watchlist** to remove them.
:::

::: step Check your file against the sample
Click **Download Sample** and compare your CSV with it. For an add, each row needs a UID, entity details and a rationale.
:::

::: step Choose the file
Drag the CSV onto the upload area, or click **Browse File** and select it. Only CSV files are accepted.
:::

::: step Upload
Click **Upload & Add**. To leave without uploading, click **Cancel**.
:::

<div class="related-modules"><div class="related-modules-title">Related in Detection &amp; Monitoring</div><div class="related-modules-grid"><a href="#name-screening" class="related-module-card"><span class="related-module-icon"><svg width="18" height="18" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><circle cx="9.5" cy="8" r="3.5"/><path d="M3.5 20a6 6 0 0 1 8-5.6"/><circle cx="16.5" cy="16" r="3"/><path d="m18.7 18.2 2.3 2.3"/></svg></span><span class="related-module-name">Name Screening</span></a><a href="#transaction-monitoring" class="related-module-card"><span class="related-module-icon"><svg width="18" height="18" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M3 12h4l2.5-6.5 5 13 2.5-6.5h4"/></svg></span><span class="related-module-name">Transaction Monitoring</span></a><a href="#typology-library" class="related-module-card"><span class="related-module-icon"><svg width="18" height="18" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="3.5" y="4" width="4.5" height="16" rx="1"/><rect x="10" y="4" width="4.5" height="16" rx="1"/><path d="m16.6 5.2 3.4-.9 3 15-3.4.9z"/></svg></span><span class="related-module-name">Typology Library</span></a></div></div>