Skip to content

Draft: Reuse ProjectPipelineJob to avoid N+1 problem

Igor Ponomarev requested to merge igor.ponomarev/ci-collate:fix-n-plus-1-pipeline-init into main

Currently initializing the CollateJob take 1 list HTTP call to get the list of ProjectPipelineJob and N number calls to fetch the ProjectJob objects. This is very inefficient and slow.

The issue with ProjectPipelineJob is that it is missing the methods like .trace() or .artifact() therefore it cannot be a replacement for ProjectJob.

Instead keep the ProjectPipelineJob and initialize a lazy ProjectJob. Change code to first check the ProjectPipelineJob for attributes and fallback to ProjectJob. Make the method calls like .trace() use ProjectJob.

This way the number of required HTTP calls will drastically go down.

Merge request reports