Forum Discussion

Norbert_Kisiel's avatar
Norbert_Kisiel
Icon for Quantexa Team rankQuantexa Team
2 hours ago
Solved

Best practice for setting runIds for ingest and batch resolver

Hi team, been a while since looking at this. We have three data sources running at different times, followed by BR. What's the best practice for setting runId for these via an automated process? Re...
  • Norbert_Kisiel's avatar
    Norbert_Kisiel
    Icon for Quantexa Team rankQuantexa Team
    2 hours ago

    The question is asking best practice for both the data ingest and the QBR.

    Your response answers the best practice for QBR, thanks!

    For the  data-ingest, we were planning to set the runId to the timestamp the job commences, essentially by supplying as a command line property to the job.

    • Rafal_Bartosz's avatar
      Rafal_Bartosz
      Icon for Quantexa Team rankQuantexa Team
      2 hours ago

      The best practice for setting a RunID is to align it to the ID of your run in your scheduler if that's possible, e.g. an airflow run id. Passing this in via a command line property is good.