stevend-hf - Hey TS team — we’re trying to use ...
# dev-help
s
Hey TS team — we’re trying to use skip_diff_check on the metadata/tml/import v2.0 endpoint to skip re-importing unchanged TML files, but we can’t get any skip behaviour out of it. The REST API v2.0 changelog (10.6.0.cl, March 2025) says: “The skip_diff_check attribute is disabled by default and can be enabled to avoid importing objects that do not have any changes.” So we expected skip_diff_check: true to skip unchanged objects. What we’re seeing — both settings produce identical results, every object UPDATE’d every time: • PROD: 4 consecutive deploys of prod_nissan using release tag v2026.04.11. Tags are immutable so source content is byte-identical. Every deploy: 19 UPDATEs, 6 UNKNOWN actions. No skips. • DEV: 2 consecutive deploys with skip_diff_check=false, identical source, ~5 mins apart. Same result, every object UPDATE’d. A few questions: 1. Does skip_diff_check actually do what the change log says? We can’t get any skip behaviour from either true or false. 2. What does the server-side diff actually compare — byte-level on the TML, parsed structure, or semantic equivalence? 3. Are there fields that get normalised/rewritten on import (GUIDs, formula parsing, list ordering) that could cause a freshly-imported TML to differ from what we sent, making the next “identical” upload look like a change? My current hypothesis is the issue is on our side — our TML renderer may not be byte-stable across runs, so your server diff sees differences even when upstream content is identical. Before we dig into our renderer though, we’d like to confirm skip_diff_check would actually help once we fix that, and what comparison logic the server applies. Thanks!
a
message has been deleted
s
You’re right to question this before spending cycles on your renderer. I could not find this in the official ThoughtSpot documentation: a documented, endpoint-level explanation of how
skip_diff_check
computes “no change”
(byte-level vs parsed vs semantic), or a list of normalized fields that affect that comparison. What I can confirm from official docs: - The REST v2 changelog page exists and includes 10.6.0.cl entries, but the retrieved sections did not include the detailed
skip_diff_check
behavior you’re asking about. - I also could not retrieve a usable REST reference node for
tmlImport
with exact request/response semantics from the tool in this session. Given that, I can’t safely assert the internal comparator logic. ### Practical guidance based on your symptoms Your observed pattern (unchanged source, repeated
UPDATE
) is consistent with one of these: 1. The server comparator is not using raw byte equality. 2. The comparator includes fields that drift between exports/imports. 3.
skip_diff_check
is currently ineffective for your object types/workflow/version path. 4. The TML you upload is not stable at the structural level even if your release artifact looks stable at a higher level. ### What to do next (high signal) 1. Run a strict A/B on the same cluster and same payload - Import exact same payload twice with
skip_diff_check=true
. - Repeat with
skip_diff_check=false
. - Capture full request + response JSON (including per-object statuses/actions). 2. Canonicalize your TML before upload - Stable key ordering, stable list ordering where order is not meaningful, normalized whitespace/newlines, consistent numeric formatting. - Hash each object payload and persist hash across deploys. 3. Round-trip check - Import once, then export immediately, canonicalize both, and diff.
- If exported form differs systematically from sent form, your next import may always look “changed” unless you render in the server’s canonical shape. 4. Escalate with a reproducible packet - Include cluster version, endpoint path, exact payload sample, two consecutive responses, and object-level action matrix (
UPDATE
/
UNKNOWN
). - Ask support/engineering specifically whether
skip_diff_check
is expected to produce a distinct “SKIP/no-op” action and for which object classes. If you want, I can help you build a minimal deterministic repro harness (single file, JS) that logs per-object hash + API response so you can hand TS support an airtight case quickly.
s
Per your earlier guidance, we tested whether our client could be sending non-byte-identical payloads across runs. Two diagnostic scripts: 1.
thoughtspot_tml
library round-trip
— loaded and dumped each of Connection / Table / Model / Liveboard fixtures twice, in-process and in separate subprocesses (with
PYTHONHASHSEED
randomised). All four types byte-identical across all runs. 2. Our full mutation pipeline — loaded each fixture, applied our deploy-time mutations (connection name stripping, GUID nulling, custom field injection, dimension formula patching), dumped to bytes, repeated. All five fixtures byte-identical across runs. Conclusion: our client produces deterministic output. If we send a TML on Monday and the same TML on Tuesday, the bytes are the same. (Diagnostic scripts available on request.)
s
On skip_diff_check semantics There's a naming issue here worth clarifying. The parameter maps to an internal flag called skip_object_import_eligibility. Setting skip_diff_check=true disables the diff-checking mechanism and forces every object to be imported unconditionally — which is consistent with what you're observing (all UPDATEs regardless of content). Setting skip_diff_check=false (the default) is what enables the diff comparison and allows unchanged objects to be skipped. We recognise the changelog description reads the opposite way. We'll flag that for correction. How the diff actually works The comparison is not byte-level against the payload you send. The server: 1. Normalises the incoming TML — primarily by injecting FQNs (GUIDs for connection, tables, join destinations, RLS references) into any fields that are absent 2. Serialises the normalised form to YAML 3. SHA-256 hashes that string to produce an ETag 4. Compares that ETag (plus the current object signature in the metadata store) against what was stored at the previous import Both values must match for an object to be skipped. Importantly, ETags are only written when skip_diff_check=false. If your previous deploys all used true, no ETags have ever been persisted for those objects, so the diff check has nothing to compare against. Why you're still seeing all-UPDATEs with skip_diff_check=false Most likely because no ETags exist yet for your objects. The sequence to get skip behaviour working: 1. Deploy with skip_diff_check=false (or omit it entirely — false is the default). Everything will UPDATE on this run, but ETags will be written. 2. Deploy again with identical source and skip_diff_check=false. Objects whose normalised TML matches the stored ETag will be skipped. Since your client output is deterministic and the normalisation (FQN injection) is stable for a given cluster state, this should produce the skip behaviour you expected once ETags are established.