ArchiTECHt Daily: Amazon embraces Apache for AI project

FWIW, this originally appeared on the website on Monday.Amazon Web Services has seemingly found open
ArchiTECHt Daily: Amazon embraces Apache for AI project
By ARCHITECHT • Issue #7
Amazon Web Services has seemingly found open source religion over the past several months, including in the field of artificial intelligence. On Monday, the cloud computing arm of Amazon announced that MXNet, its framework of choice for building deep learning systems, has been accepted into the Apache Incubator program.
This is a good move on the part of AWS for a few reasons:
  1. Open source is a great way to ramp up innovation on a project by exposing it to a greater number of users and contributors. While MXNet had always been open source, contributing it the ASF should increase its exposure and bring a broader community of users and contributors on board.
  2. Contributing MXNet to ASF helps mitigate (but not eliminate) any air of vendor bias over the project. This is important for AWS because …
  3. Its biggest competitors in cloud computing, Microsoft and Google, both have open source deep learning frameworks of their own. Both companies also have better and longer-standing reputations in the AI and open source communities. For what it’s worth, though, neither Microsoft’s CNTK nor Google’s TensorFlow are ASF projects.
However, as I wrote earlier this month, much of the innovation in AI is already happening in the open—MXNet included—so its ultimate status as an Apache project probably doesn’t change a whole lot. In many spaces, including deep learning, open source is just table stakes to attracting any meaningful interest in a project. (Amazon has made some notable open source moves elsewhere, too.)
In the end, it’s the cloud computing part of this that really matters. As I explained around CES time, there’s a boatload of money to be made by capitalizing on consumer demand for “smart” devices, from hairbrushes to Amazon Echoes to household robots. For companies like AWS, Google and Microsoft, that money comes from selling devices themselves, but also from selling the cloud services to help other companies build their own devices.
Like all platform plays, there’s a virtuous cycle in play for cloud providers. The more consumers using a particular device, the more appealing it is for developers to integrate with that device. So the more appealing it is to use a particular provider’s cloud to build that integration.
AI is just the latest (and possibly biggest) battleground for cloud providers. AWS might not have too strong a reputation in the open source or AI circles yet, but it has build itself one heck of a platform. It knows what it must do to maintain its edge in the AI era. Only a brave man would bet against it.

Around the web: All things data
Research into how we’ll optimize data processing and data management for the Internet of Things is picking up, mostly focused on edge computing. On a related note, check out this paper about benchmarking stream processing frameworks for IoT workloads.  •  Share
A good take on the state of Hadoop security, which features a bunch of open source projects primarily driven by competing vendors. Anyone else sense we’ve seen the end of OSS projects like Hadoop that spawn entire markets?
Hortonworks actually announced SmartSense, a paid service for optimizing Hadoop cluster performance, a year and a half ago, but I missed it. You might have, too. Anyhow, moving further away from HDP support is a good thing.
Around the web: Cloud and infrastructure
This is an interesting look at how traffic flows at Facebook within and across racks, clusters and data centers. And if you’re into the next-gen networking infrastructure that’s handling this movement, check out this collection of network engineering presentations from a recent Facebook event.
This is the second post in a four-part series about serverless computing from a former founder. Lots of good info on what it is and how to do it. And if you want a deep dive on a new serverless framework for Kubernetes, click here.  •  Share
Speaking of Kubernetes … StatefulSets for running stateful services on Kubernetes is still in beta, but it appears to be advancing quickly. For many applications, stateless alone will not cut it.
Yeah, but what’s life without a little microservices-washing? Actually, this is useful advice for newcomers who want to avoid getting taken for a ride. 
Around the web: Artificial intelligence
Google uses deep learning for many, many things, including keeping the Play store free of spam and malicious apps.
This post does a good job breaking down some various schools of thought in AI research. Also, a master algorithm for the web seems both very convenient and very disturbing.
Apparently, there is an industry devoted to defending encrypted networks against quantum computing attacks. Others argue quantum computing will be a boon to security. Both can be true.
You know things are going good when you can, “It’s just another study where AI is advancing cancer research.” Between AI, gene editing and a gut of computing resources, the state of oncology has arguably never looked better.
This research paper does a good job analyzing possible (and maybe not so possible) ethical issues arising from advances in AI. One area it covers, bias in machine learning models, is already a problem.  •  Share
Somewhere between TensorFlow and Watson lies Bonsai, which its teaching-based language called Inkling. It’s a smart idea, and the more power to the company if it can get heard above the noise from bigger players.
Did you enjoy this issue?
The most interesting news, analysis, blog posts and research in cloud computing, artificial intelligence and software engineering. Delivered daily to your inbox. Curated by Derrick Harris. Check out the Architecht site at
Carefully curated by ARCHITECHT with Revue. If you were forwarded this newsletter and you like it, you can subscribe here. If you don't want these updates anymore, please unsubscribe here.