ArchiTECHt Daily: Yahoo Engineering keeps on keepin' on

Yahoo might be the world's slowest sinking ship, but don't blame the engineers. They're still doing e
ArchiTECHt Daily: Yahoo Engineering keeps on keepin' on
By ARCHITECHT • Issue #17
Yahoo might be the world’s slowest sinking ship, but don’t blame the engineers. They’re still doing everything they can to keep it afloat. Or, at least, to maintain whatever reputation it has left as a home of innovation and useful products.
On Monday, for example, Yahoo Engineering open sourced a deep learning framework called TensorFlowOnSpark. In terms of utility for organizations with a big data pipeline already in place, this is hard to beat. It lets users train deep learning models using Google’s popular TensorFlow framework, using the even more popular Apache Spark as the compute layer, using the even more popular Hadoop Distributed File System as a unified storage layer.
Never forget: Yahoo is the reason we have Hadoop, which created a whole ecosystem of new big data technologies (including Spark) and helped companies such as Facebook scale into giants.
TensorFlowOnSpark is not Hadoop, but that doesn’t mean it can’t also be useful. It’s the kind of creation only possible by a company with robust enough traffic to identify some edge cases, but also with enough legacy infrastructure in place (and, let’s assume, a constrained enough budget) that building out an entirely new system seems like overkill. Especially with regard to the latter point, thanks in part to Hadoop’s considerable footprint, there are a lot of those companies around. 
Also on Monday, Yahoo detailed its new backend architecture for Flickr, called Tripod. It represents a move away from Flickr’s monolithic legacy and into microservices, utilizing lots of Yahoo-developed databases, big data technologies and computer vision algorithms in the process. 
Yahoo will never again be anywhere near Google, Facebook or even Twitter, probably, but you have to hand it to the company’s engineering team. They’re still building, still committed to open sources and, possibly, making it easier for some mere mortal companies to keep up with ceaseless infrastructure innovation.

Around the web: Cloud and infrastructure
It’s not clear that artificial intelligence can fundamentally change the nature of sales and CRM software, but Salesforce—and a seemingly endless parade of startups—is betting it can.
(I sense a patter here …) Lots of new Watson-based security products, and even a voice assistant for security. The latter seems less useful than Watson’s pattern recognition might be.
Four new services for dealing with IoT data. Oracle isn’t wasting time (well, any more than it already has) on this cloud thing.
I thought we were done with this discussion years ago (They can’t!) but this is a decent analysis of the situation. 
From a company called Serverless. So meta, and also potentially useful for folks tired of dealing with Jira when they don’t have to.
This is good advice for how to think about data management and processing in the ultimate stateless scenario.
The results of the DeepMind study (below)
Around the web: Artificial intelligence
… or in competition with one another, in order to solve problems or earn a reward. This is both a fascinating and, I’ll admit, a little creepy application of reinforcement learning. Here’s the paper:
With a technique they call hybrid code networks, the researchers claim they can achieve equal performance with less training data.  •  Share
Uh, doi. They’re still much better than when Siri first came out, but with the pace of AI research we should see advances fairly quickly.
… MIT researchers have developed a low-power speech recognition chip that could be embedded into rather small devices.
It already is in some cases it seems, with quants being replaced by machine learning experts and algorithms. You have to think this will be a huge area for AI in the years to come.
Last month, I wrote about how mundane uses of deep learning are actually exciting because it means the tech is going mainstream. Here’s another example of that.
This is some hardcore theoretical physics research, but the applications could potentially be useful in fields like pharmaceuticals or others with lots of uncertainty and complex data.  •  Share
Around the web: All things data
A Indian company called Playment is letting companies crowdsource certain data analysis tasks to an on-demand workforce. It’s not the first time we’ve seen this approach, but using mobile phones could make it much faster and more appealing for everyone.
Did you enjoy this issue?
The most interesting news, analysis, blog posts and research in cloud computing, artificial intelligence and software engineering. Delivered daily to your inbox. Curated by Derrick Harris. Check out the Architecht site at
Carefully curated by ARCHITECHT with Revue. If you were forwarded this newsletter and you like it, you can subscribe here. If you don't want these updates anymore, please unsubscribe here.