Max Kelsen is only one of three Australian companies to attain AWS’s machine learning competency status, and was named AWS’s 2020 Partner of the Year for Data, Analytics and Machine Learning. We are constantly in the thick of upskilling our team by attending conferences that fuel our thirst for knowledge and keeping our curiosity alive.

AWS re:Invent is a free 3-week virtual conference held by AWS as an opportunity for AWS experts to lead sessions and share the latest news, technologies, and trends within the space. The Max Kelsen team are attending for the entirety of the conference and sharing with you a weekly summary of the key sessions that touch upon the components and sub-services that we use, what we are excited about and why these are important.

#1 — Amazon Braket Updates — PennyLane Support & Tensor Network Simulators

Session Introduction:

Similar to the way that CPUs and GPUs work hand-in-hand to address large scale classical computing problems, the emerging field of hybrid quantum algorithms joins CPUs and QPUs to speed up specific calculations within a classical algorithm. This allows for shorter quantum executions that are less susceptible to the cumulative effects of noise and that run well on today’s devices. This session talks about 2 updates:

  1. Making the PennyLane library available so that you can build hybrid quantum-classical algorithms and run them on Amazon Braket.
  2. The AWS Center for Quantum Computing is working to address the noise issue in two different ways: we are investigating ways to make the gates themselves more accurate, while also working on the development of more efficient ways to encode information redundantly across multiple qubits.

Access the full recording here.

Max Kelsen Summary:

The Braket offering from AWS offers an impressive selection of exciting quantum platforms to test out. The PennyLane integration into this service provides the capabilities to construct hybrid quantum-classical algorithms in a differentiable way, allowing you to “follow the gradient”. This parallels classical machine learning frameworks like TensorFlow and PyTorch, an exciting step forward for the field of quantum machine learning in which Max Kelsen’s research team is focusing on. Our team are actively looking to use this platform in upcoming research projects furthering our work in the application of machine learning in quantum computing.

The expanded choice of simulators offered also boast an impressive extension to classical quantum simulators, with the tensor network-based simulator being able to simulate up to 50 qubits for specific applications. This remarkable number of simulated qubits opens the door for many novel proof of concept quantum applications being showcased and tested, and will significantly boost not only Max Kelsen’s capabilities in quantum algorithm design, but other leading organisations within this space.

#2 — VPC Reachability Analyzer

Session Introduction:

With Amazon Virtual Private Cloud (VPC), you can launch a logically isolated customer-specific virtual network on the AWS Cloud. As users expand their footprint on the cloud and deploy increasingly complex network architectures, it can take longer to resolve network connectivity issues caused by misconfiguration. AWS announces VPC Reachability Analyzer, a network diagnostics tool that troubleshoots reachability between two endpoints in a VPC, or within multiple VPCs. Access the full recording here.

Max Kelsen Summary:

We find that this will be a really helpful tool in uplifting the quality of life for our teams. Our teams (and our customers!) often have to deploy solutions within existing complex VPC environments, and this tool will greatly streamline the troubleshooting process — no more needing to spin up a jump box to run network traces!

#3 — AWS C6gn Graviton

Session Introduction:

AWS announced an addition to our broad Arm-based Graviton2 portfolio with C6gn instances that deliver up to 100 Gbps network bandwidth, up to 38 Gbps Amazon Elastic Block Store (EBS) bandwidth, up to 40% higher packet processing performance, and up to 40% better price/performance versus comparable current generation x86-based network optimised instances. Access the full recording here.

Max Kelsen Summary:

We initially missed this announcement last week, however this tweet has put it back on our radar!

Image from Twitter account Matthew S. Wilson @_msw_

It is exciting to see the continued and rapid investment by AWS in their Graviton processors. The announcement of the C6gn instances, with greatly improved networking performance represents another leap forward, and we are particularly interested in the higher end of the new instance sizing (especially the C6gn — 16x larger with support for the Elastic Fabric Adaptor). We work with a number of our customers to run their bioinformatics workloads in the cloud. Benchmarks suggesting Graviton can have up to 30–50% price/performance over traditional x86 are incredibly exciting to see — and represent the opportunity for major savings when running bioinformatics pipelines at scale.

#4 — Sagemaker Data Parallelism

Session Introduction:

Amazon SageMaker now supports a new data parallelism library that makes it easier to train models on datasets that may be as large as hundreds or thousands of gigabytes. Amazon SageMaker now helps ML teams reduce distributed training time and cost, thanks to the SageMaker Data Parallelism (SDP) library. Available for TensorFlow and PyTorch, SDP implements a more efficient distribution of computation, optimises network communication, and fully utilises our fastest p3 and p4 GPU instances. Access the full recording here.

Max Kelsen Summary:

Improvements to GPU utilisation when operating across multiple GPU’s (and nodes) are always welcome! In our experience, teams often battle to tune scaled-out training workloads — maintaining high utilisation of all GPUs across a cluster doesn’t always come easy. With the introduction of the SageMaker Data Parallelism API, a drop in (in most cases!) replacement for PyTorch’s Distributed Data Parallel APIs, users are provided with the ability to leverage SageMaker’s distributed training optimisations without needing to refactor existing work. We are very keen to see the improvements to distributed training — increased GPU utilisation means reduced training time and most importantly, reduced costs.

#5 — Amazon HealthLake

Session Introduction:

AWS announced the launch of Amazon HealthLake, a fully managed, HIPAA-eligible service, now in preview, that allows healthcare and life sciences customers to aggregate their health information from different silos and formats into a centralised AWS data lake. HealthLake uses machine learning (ML) models to normalise health data and automatically understand and extract meaningful medical information from the data so all this information can be easily searched. Then, customers can query and analyse the data to understand relationships, identify trends, and make predictions. Access the full recording here or read more on the blog here.

Max Kelsen Summary:

Amazon HealthLake is a managed, Fast Healthcare Interoperability Resources (FHIR) compatible and HIPAA-eligible datastore and analysis platform. We see customers continually seeking improved analysis, deeper integration and greater flexibility across their data-generating medical and life sciences systems. To date, to achieve this within AWS, whilst there are useful templates to develop FHIR compatible interfaces, this has required a non-trivial amount of engineering effort. So with Amazon HealthLake, it is great to see AWS providing FHIR compatible interfaces for integrating existing systems with the data store, as well as providing the ability to tag and restructure unstructured data with the FHIR specification. Once data is ingested, the ability to query data via simple search interfaces, or perform deeper analysis within SageMaker Notebooks are fantastic additions. We are excited to see the opportunities (especially for machine learning) that HealthLake presents to our customers through continued use.

#6 — ECR Cross Region Replication

Session Introduction:

Replicating container images across regions in Amazon Elastic Container Registry (ECR) automatically has been one of the most asked features and Amazon has finally shared the good news: it has landed. Where previously you had to implement the replication yourself you can now leave the heavy lifting to AWS and focus on building and running your applications. In this session, AWS explain how the Cross Region Replication (CRR) feature works in ECR and how you can start benefiting from it. Access the full recording here.

Max Kelsen Summary:

Much like the release of the VPC Reachability Analyzer, the introduction of cross region replication for ECR is another great quality of life improvement for our developer teams — removing internal processes and tooling we had developed and maintained to achieve the same outcomes. With cross region replication for ECR, it is now trivial to ensure our containers are distributed as close as possible to our multi-region compute workloads.

#7 — Redshift Data Sharing

Session Introduction:

Amazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that makes it simple and cost-effective to analyse all your data using standard SQL. Tens of thousands of customers use Amazon Redshift to process exabytes of data per day and power analytics workloads such as high-performance business intelligence (BI) reporting, dashboarding applications, data exploration, and real-time analytics. AWS launched their new Amazon Redshift data sharing capability, which enables users to securely and easily share live data across Amazon Redshift clusters. Access the full recording here.

Max Kelsen Summary:

Max Kelsen works with a number of enterprises to build out their data lakes and associated analytical capability — and more often than not, Redshift is the tool of choice for providing access to data from their lake to BI and analytics users. With the introduction of Redshift Data Sharing, the ability to support multi-tenant architectures in Redshift is greatly improved. The ability to ensure the data being presented to consumers is up-to-date, as well as managing permissions, SLAs and cost-attribution will be a huge win for our customers using Redshift at scale.

#8 — SageMaker Neo — improvements & additional hardware platform support

Session Introduction:

Amazon SageMaker Neo enables developers to train machine learning (ML) models once and optimise them to run on any Amazon SageMaker endpoints in the cloud and supported devices at the edge. Over the past few months, Neo has added a number of key new features:

  • Expanded support for PC and mobile devices.
  • Heterogeneous execution with NVIDIA TensorRT.
  • Bring Your Own Codegen (BYOC) framework.
  • Inference optimised containers.
  • Compilation for dynamic models.

In this session, AWS summarise how these new features allow you to run more models on more hardware platforms both faster and more efficiently. Access the full recording here.

Max Kelsen Summary:

Increasingly, machine learning models are being deployed on edge devices, and even mobile devices, providing users with a more interactive experience through reduced latency, as well as often being able to leverage compute that would otherwise be under-utilised at the edge. Over the coming 12 months, Max Kelsen anticipates that we will see the focus on edge inference continue to grow, supported by improvements to the development and deployment tooling provided by services like SageMaker Neo. It is great to see the SageMaker Neo services continually and rapidly expand its supported architectures and devices, whilst improving the developer experience for compiling models for edge and mobile compute.

And that’s a wrap for week two! Please stay tuned as we provide the last weekly update of the third and final week of the AWS re:Invent conference, and do not be afraid to reach out to us at hello@maxkelsen.com if you would like to start a conversation about any of the above topics and updates.

We are an Artificial Intelligence and Machine Learning consultancy that delivers competitive advantage for government and enterprise. https://maxkelsen.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store