• Tvrtko Ursulin's avatar
    igt/media-bench.pl: Media workload analyzer · 0943be89
    Tvrtko Ursulin authored
    The high level goal of this script is to programatically analyze
    the simulated media workloads (gem_wsim) by finding an optimal
    load balancing strategy, and also detecting any possible
    shortcomings of the same.
    
    When run without command line arguments script will run through
    both of its phases.
    
    In the first phase it will be running all the known balancers
    against the all the known workloads, and for each combination
    look for a point where aggregated system throughput cannot be
    increased by running more parallel workload instances.
    
    At that point each balancer gets a score proportional to the
    throughput achieved, which is added to the running total for the
    complete phase.
    
    Several different score boards are kept - total throughput, per
    client throughput and combined (total + per client). Weighted
    scoreboards are also kept where scores are weighted based on the
    total variance detected for a single workload. This means scores
    for workloads which respond well to being balanced will be worth
    more than of the ones which do not balance well in neither of
    the configurations.
    
    Based on the first phase a "best" balancing strategy will be
    selected based on the combined weighted scoreboard.
    
    Second phase will then proceed to profile all the selected
    workloads with this balancer and look at potential problems with
    GPU engines not being completely saturated.
    
    If none of the active engine is saturated the workload will be
    flagged, as it will if the only saturated engine is one of the
    ones which can be balanced, but the other one in the same class
    is under-utilized.
    
    Flagged workloads then need to be analyzed which can be achieved
    by looking at the html of the engine timelines which are
    generated during this phase. (These files are all put in the
    current working directory.)
    
    It is quite possible that something flagged by the script as
    suspect is completely fine and just a consequence of the
    workload in question being fundementally unbalanced.
    
    It is possible to skip directly to the second phase of the
    evaluation by using the -b command line option. This option must
    contain a string exactly as understood by gem_wsim's -b option.
    For example '-b "-b rtavg -R"'.
    
    Apart from being run with no arguments, script also supports a
    selection of command line switches to enable fine tuning.
    
    For example, also including the complete output from the script
    in order to be more illustrative:
    
    -8<---
    + scripts/media-bench.pl -n 642317 -r 2 -B rand,rtavg -W media_load_balance_hd12.wsim,media_load_balance_fhd26u7.wsim
    Workloads:
      media_load_balance_hd12.wsim
      media_load_balance_fhd26u7.wsim
    Balancers: rand,rtavg,
    Target workload duration is 2s.
    Calibration tolerance is 0.01.
    Nop calibration is 642317.
    
    Evaluating 'media_load_balance_hd12.wsim'... 2s is 990 workloads. (error=0.00750000000000006)
      Finding saturation points for 'media_load_balance_hd12.wsim'...
        rand balancer ('-b rand'): 6 clients (1412.576 wps, 235.429333333333 wps/client).
        rand balancer ('-b rand -R'): 6 clients (1419.639 wps, 236.6065 wps/client).
        rtavg balancer ('-b rtavg'): 5 clients (1430.143 wps, 286.0286 wps/client).
        rtavg balancer ('-b rtavg -H'): 5 clients (1339.775 wps, 267.955 wps/client).
        rtavg balancer ('-b rtavg -R'): 5 clients (1386.384 wps, 277.2768 wps/client).
        rtavg balancer ('-b rtavg -R -H'): 6 clients (1365.943 wps, 227.657166666667 wps/client).
      Best balancer is '-b rtavg'.
    
    Evaluating 'media_load_balance_fhd26u7.wsim'... 2s is 52 workloads. (error=0.002)
      Finding saturation points for 'media_load_balance_fhd26u7.wsim'...
        rand balancer ('-b rand'): 3 clients (46.532 wps, 15.5106666666667 wps/client).
        rand balancer ('-b rand -R'): 3 clients (46.242 wps, 15.414 wps/client).
        rtavg balancer ('-b rtavg'): 6 clients (61.232 wps, 10.2053333333333 wps/client).
        rtavg balancer ('-b rtavg -H'): 4 clients (57.608 wps, 14.402 wps/client).
        rtavg balancer ('-b rtavg -R'): 6 clients (61.793 wps, 10.2988333333333 wps/client).
        rtavg balancer ('-b rtavg -R -H'): 7 clients (60.697 wps, 8.671 wps/client).
      Best balancer is '-b rtavg -R'.
    
    Total wps rank:
    ===============
      1: '-b rtavg' (1)
      2: '-b rtavg -R' (0.989191465637926)
      3: '-b rtavg -R -H' (0.973103630772601)
      4: '-b rtavg -H' (0.938804458876241)
      5: '-b rand -R' (0.874465740398305)
      6: '-b rand' (0.874342391093453)
    
    Total weighted wps rank:
    ========================
      1: '-b rtavg -R' (1)
      2: '-b rtavg' (0.998877134022041)
      3: '-b rtavg -R -H' (0.982849160383224)
      4: '-b rtavg -H' (0.938950446314292)
      5: '-b rand' (0.80507369080098)
      6: '-b rand -R' (0.80229656623594)
    
    Per client wps rank:
    ====================
      1: '-b rtavg -H' (1)
      2: '-b rand' (0.977356849770376)
      3: '-b rand -R' (0.976222085591368)
      4: '-b rtavg' (0.888825068013012)
      5: '-b rtavg -R' (0.875653417817828)
      6: '-b rtavg -R -H' (0.726389466714194)
    
    Per client weighted wps rank:
    =============================
      1: '-b rand' (1)
      2: '-b rand -R' (0.996866139192282)
      3: '-b rtavg -H' (0.986348733324348)
      4: '-b rtavg' (0.811593544774355)
      5: '-b rtavg -R' (0.805704548552663)
      6: '-b rtavg -R -H' (0.671567075453688)
    
    Combined wps rank:
    ==================
      1: '-b rtavg' (1)
      2: '-b rtavg -R' (0.989191465637926)
      3: '-b rtavg -H' (0.972251783752137)
      4: '-b rtavg -R -H' (0.949708930404222)
      5: '-b rand' (0.914594701126905)
      6: '-b rand -R' (0.914312395840401)
    
    Combined weighted wps rank:
    ===========================
      1: '-b rtavg' (1)
      2: '-b rtavg -R' (0.995945739226824)
      3: '-b rtavg -H' (0.984347862855008)
      4: '-b rtavg -R -H' (0.956920992185625)
      5: '-b rand' (0.899001713089319)
      6: '-b rand -R' (0.896984246540919)
    
    Balancer is '-b rtavg'.
    Idleness tolerance is 2%.
    
    Profiling 'media_load_balance_hd12.wsim'...
          2s is 992 workloads. (error=0.00150000000000006)
          Saturation at 6 clients (1434.207 workloads/s).
        Pass [ 0: 0.57%, 2: 22.59%, 3: 23.30%, ]
    
    Profiling 'media_load_balance_fhd26u7.wsim'...
          2s is 52 workloads. (error=0.001)
          Saturation at 6 clients (61.823 workloads/s).
        WARN [ 0: 7.77%, 2: 0.66%, 3: 28.70%, ]
    
    Problematic workloads were:
       media_load_balance_fhd26u7.wsim -c 6 -r 52 [ 0: 7.77%, 2: 0.66%, 3: 28.70%, ]
    
    -8<---
    Signed-off-by: Tvrtko Ursulin's avatarTvrtko Ursulin <tvrtko.ursulin@intel.com>
    0943be89
Makefile.am 111 Bytes