Hi All, we faced an `issue` where the the element...
# community-support
w
Hi All, we faced an
issue
where the the elementary report didn't show the test for sources I have searched the all slack threads, but can't find the answer. Can someone give us a clue on where we might set it wrong? Thanks! ---------------------------------------------------------- For Example: • We have a source table:
fivetran_database.google_sheets.segment_malaysia_segment_mapping_list
• We have set a
not_null
test for the column:
_FIVETRAN_SYNCED
◦ Here is the code
Copy code
- name: _FIVETRAN_SYNCED
            tests:
              - not_null
• In the job: dbt test, we saw the test was executed • But we can't see the test in the report (attached) ---------------------------------------------------------- Not only the above example, actually all the tests for the sources were not showed in the report. Can someone give us a clue where we might set the config wrong? Thanks!
i
Hey @WEIHAO LI, thank you for reaching out! Can you share with me you dbt.log that includes the run of the dbt test over your source?
w
Hi @Idan Shavit! Really appreciate the reply Absolutely yes! attached is the log of the dbt test. Not sure whether this info is useful, but also provide the report and the data of ELEMENTARY_TEST_RESULTS Please let me know if anything might help us to find some clues. Thanks!
Hi @Idan Shavit Not sure the information I provided is enough for investigating? If there is any information that might useful, please do let me know. It's also a big help if you can identify some potential causes based on your experience, I can go test them out. Thank you!
n
Hey @WEIHAO LI! Thanks for all the information this is really helpful! Just one last thing that might help - can you share more of your configuration file? (Where you configure the source and how the tests are structured)
From the logs it does seem like no tests are running for
fivetran_database.google_sheets
and I want to make sure how the configuration looks like
w
Hi @Noa Kurman Definitely yes!! below is the config for the source and its test. In logs, the source name for the source table:
fivetran_database.google_sheets.segment_malaysia_segment_mapping_list
is
fivetran_segment_SEGMENT_MALAYSIA_SEGMENT_MAPPING_LIST
. Also attached the snapshot for a run test in the logs
Copy code
version: 2

sources:
  - name: fivetran_segment
    database: FIVETRAN_DATABASE
    schema: GOOGLE_SHEETS
    loader: fivetran
    description: |
      - This source includes mapping list of the segment

    tables:
      # table = SEGMENT_MALAYSIA_SEGMENT_MAPPING_LIST
      - name: SEGMENT_MALAYSIA_SEGMENT_MAPPING_LIST
        description: |
          the segment mapping list for Malaysia 
          [source url](<https://docs.google.com/spreadsheets/d/1lDGFGTxktvVbw6Rlg2Lp5T5VG5B3C-YXGRn3guFrDN8/edit#gid=0>)
        meta:
          owner: weihao.li@partipost.com
          fivetran:
            name: google_sheets.segment_malaysia_segment_mapping_list
            source: 
              type: Google Sheets
              url: |
                <https://docs.google.com/spreadsheets/d/1lDGFGTxktvVbw6Rlg2Lp5T5VG5B3C-YXGRn3guFrDN8/edit#gid=0>
        loaded_at_field: _FIVETRAN_SYNCED

        columns:
          - name: email
            description: "{{ doc('pd_auth_user_EMAIL') }}"
            tests:
              - not_null
              # the email should be the Paritposter auth email address
              - relationships:
                  to: source('fivetran_pd_auth', 'auth_user')
                  field: email
            # To provide the flexibility to add segments in the future, the segment name validation was built directly in the G-sheet
            # url: <https://docs.google.com/spreadsheets/d/1lDGFGTxktvVbw6Rlg2Lp5T5VG5B3C-YXGRn3guFrDN8/edit#gid=1247391011>
          - name: segment_name_a
            description: "{{ doc('stg_profile_segments_segment_name') }}"
          - name: segment_name_b
            description: "{{ doc('stg_profile_segments_segment_name') }}"
          - name: _FIVETRAN_SYNCED
            description: "{{ doc('fivetran_synced') }}"
            tests:
              - not_null
          - name: _ROW
            description: "{{ doc('fivetran_row') }}"
            tests:
              - not_null
              - unique
n
Interesting thanks again for all the information! So now I understand the name of the source, I can’t find it in the table (elementary_test_results) - am I wrong? Is it possible you ran it with a different profile, or didn’t run it with elementary somehow? Cause as you say we do see this test has been running but the result did not get to the right table
Also can you please send you dbt_sources table? (I know I promised the previous one was the last!) We suspect change to the source name might be the issue here so would like to see the sources