asdasd
The choice of batch size is another important parameter to consider for materials scientists. It can be tuned when attempting to scale up for HTE. A larger batch size is usually ideal since it provides higher throughput, and thus more time savings since lesser iterations are required. However, batch size affects the performance of optimisation strategies, potentially reducing the number of iterations needed in a run. We thus perform optimisation on the same synthetic problems for different batch sizes, keeping dimensionality at dim=8 and with the same evaluation budget of 192 points and 10 runs as mentioned earlier.
The authors of qNEHVI hypothesised that it operates better at small batch sizes by providing a smoother gradient descent in sequential optimisation. Results reported in Figure 6 a), b) and d) for ZDT1, ZDT2 and MW7, respectively, support this hypothesis, and we clearly observe that the lowest batch size setting of 2, as represented by the pink line, has the best performance overall. Interestingly, this is also the case for U-NSGA-III where the lowest batch size of 2 tends to give better HV for ZDT1-3 as seen by the blue line. This is also empirically shown in literature where, given a total budget, higher populations may impede convergence as it effectively limits the number of iterations.[86–88]
It is suggested that the same did not apply for MW7 since the disconnected PF was often not fully explored due to differences in initilisation and how the heuristic search operated, which we discuss previously for Figure 4 and 5. Instead, a larger batch size i.e. larger population is beneficial in maintaining solutions across disconnected regions of objective space, as seen by the red line in Figure 6d). We also explain why this did not apply to ZDT3: since the initial sampling was generally able to cover the search space well, there are relatively little ‘lost’ regions as seen from Figure 4c). Additionally, we provide optimisation trajectory plots for U-NSGA-III at different batch sizes in the SI 3 to illustrate this.
Furthermore, we also observe that qNEHVI has greater variance in log HV difference, compared to U-NSGA-III. This further reinforces our hypothesis that the performance of qNEHVI is in part due to the stochastic QMC sampling, whilst the heuristic nature of U-NSGA-III means that the evolution of solutions is more consistent.