As part of our ongoing work to run DeepCell on GCP, I’m pleased to share our benchmarking results using Google Batch’s private preview feature: node pools.
Node pools allow Batch to reuse infrastructure across jobs, which can improve end-to-end performance for jobs with a lot of setup time (like our 3 GB container). Using a node pool reduced our benchmark time by 80%. 😎
https://medium.com/@dchaley/running-deepcell-on-google-batch-with-node-pools-8fe7d1c121a2
