Your data needs to keep up with the state of play today. Event-driven architecture will get you up to speed


The Pop-Up Playground Party at AWS re:Invent 2019, sponsored by MongoDB and Confluent, was an exclusive, interactive event featuring immersive art, light, gaming, and play activations with an open bar, DJs, and more, at the Industrial Las Vegas.

Attendees got to create unforgettable memories (or at least killer Instagram photos) at the glowing ball pit, DIY graffiti walls, retro arcade games, and other immersive activities, such as getting a temporary tattoo in a sports car. Meanwhile, the Hood Internet and Vegas’s hottest DJs kept the soundtrack fresh all night long!

In a word, the Pop-Up Playground was: fun.

Something that is way less fun: being one of the many companies that increasingly find it impossible to keep up with the pace of business because their data platforms can’t keep up. Under those circumstances life is anything but a party, in fact, the opposite of a party. 

That’s why MongoDB and Confluent teamed up, not just to throw the best party at re:Invent but to help companies run their businesses in real time. 

An effective approach to real time could not come at a better time, because many companies are still trying to meet their data needs in ways that don’t match modern business reality. Some are mired in 40-year-old relational database technology. Others think updating to a NoSQL database is positioning them for the future. They soon learn that while getting data in may be easy enough, getting insights out is a whole different story, and what they are actually positioned for now are niche applications, not an effective new data strategy.

The results of pursuing the wrong data strategy can be harmful or fatal. CNBC Markets found that the average lifespan of S&P 500 companies has fallen from nearly 60 years in the 1950s to less than 20 years today. Innosight modeled the rate of attrition from the S&P 500 and predicted that over the next 10 years, half the companies will be replaced: drop out, go out of business, or be acquired, as digital transformation lowers barriers to entry for new players and helps drive out incumbents trapped in legacy infrastructures and processes.

The traditional request-driven data architecture that still exists at many companies helps keep them trapped in legacy limitations. It requires users and applications to make requests and wait until the requested information becomes available. Waiting for data kills opportunity and agility.

By contrast, event-driven architecture, which I posted about on,  proactively makes a stream of data from source systems (producers) available in real time. Consuming applications and services (consumers) subscribe to topics of interest and consume data at their own pace. Capturing and acting on events in real time enables systems to react automatically and immediately to events. This helps a company rapidly position itself to outflank competitors. Detecting operational errors lets it take immediate corrective action. These benefits translate into not only operational excellence and cost savings but also enhanced customer experience as the company optimizes customer-facing processes. More broadly, an event-driven architecture helps the organization improve business agility.

The key driver today in making the need for real-time data an organizational imperative is the emergence of microservices. Microservices architecture breaks up monolithic applications into small, discrete services or functions. It creates self-sufficient sprint teams empowered to bring new capabilities online independently of each other, then over time evolve and upgrade their microservice without impacting adjacent microservices. That is the essence of agility.

As beneficial as microservices can be, though, they require the ability to work with large volumes of data that change frequently, which is a challenge many existing systems cannot meet. Trying to implement microservices in a legacy relational database incurs the pain and friction of having to define a schema in the database and re-implement that same schema again to effect object-relational mapping (ORM) at the application layer. Then your development team has to repeat the process, first for each microservice and then every change to the data model as application functionality evolves.

With MongoDB, data modeling for microservices is easy, which is a big reason MongoDB is at the core of many event-driven systems today. MongoDB’s flexible document model gives you the best way to work with data, lets you intelligently place data where you need it (and when, as in immediately), and gives you the freedom to run anywhere. MongoDB helps you move at the speed your users demand. It gives you the power to launch new digital initiatives and bring modernized applications to market faster, running reliably and securely at scale, unlocking insights and intelligence ahead of your competitors.


By starting with the core MongoDB data platform and binding in complementary technologies, MongoDB provides the data persistence heart of an event-driven architecture. MongoDB and Confluent work together to enable you to readily build microservices and event-driven architectures to become an agile organization.

Confluent Platform, including Apache® Kafka® and Kafka Connect, is designed as an event messaging queue for massive streams of data that sequentially writes events into commit logs, allowing real-time data movement between your services and data sources. The MongoDB Connector for Apache® Kafka® — developed and supported by MongoDB engineers, and verified by Confluent as a first-class component of Confluent Platform — simplifies building robust, reactive pipelines to move events between systems. You can use MongoDB as a sink (consumer) to ingest events from Kafka topics directly into MongoDB collections, exposing the data to your services for efficient querying, enrichment, and analytics, as well as for long-term storage. You can also use MongoDB as a source (producer) for Kafka topics; in this mode, data is captured via Change Streams within the MongoDB cluster and published straight into Kafka topics. These capabilities enable consuming apps to react to data changes in real time.

MongoDB-powered event-driven architectures are at work in a range of user cases including IoT and other time series applications; financial services; AI; predictive maintenance, primarily in manufacturing but also in other verticals; Web activity tracking and log aggregation; and as an operational data layer (ODL) integrating and organizing siloed enterprise data to make it available to all users and consuming apps. Customers who have deployed event-driven architecture powered by MongoDB include Ticketek, EG,, Man AHL, and

The figure below shows MongoDB and Confluent working together in an event-driven architecture supporting a microservices-based e-commerce application.

In this scenario, fuel costs to ship some items have just gone up, which could impact pricing. This produces events about the cost increase and places them into Apache Kafka. The Pricing microservice consumes the event, analyzes it against existing data, and produces events conveying the new pricing. MongoDB Atlas captures this data and, through the MongoDB Connector for Apache Kafka, publishes it into Kafka topics, which makes the data available to all consumers. Microservices directly impacted by pricing changes, such as those that manage inventory, marketing, promotions & coupons, point of sale (POS), and the e-commerce provider’s order management system (OMS), consume the price change events and update their individual databases accordingly. MongoDB Atlas aggregates and persists data from all microservices, enriches event streams with data from other sources, including historical data, and provides a central repository. This enables applications and users to benefit from all data across all microservices and provides a unified view of state across the e-commerce provider’s enterprise.

To learn more, download our white paper, How MongoDB Enables Real-Time Data with Event-Driven Architecture, and visit our Real-Time Analytics page.

I’m now at MongoDB!


I am now a proud member of the Product Marketing team at MongoDB: the first database company to go public in more than two decades (NASDAQ: $MDB), its business growing at ~50% YoY, with the technology and vision to take on the multibillion-dollar incumbents as it disrupts and reshapes an entire industry. As an analyst I had forecast a total market opportunity of $67.89 billion in big data and analytics by 2019, growing to nearly $111 billion by 2022. I am excited to be at a truly global company capturing a sizable and growing share of that opportunity!

My role is Senior Solutions Marketing Manager, a core part of an energetic and globally distributed team reporting to the Senior Director, Products and Solutions, based in the UK. I am responsible for driving solutions marketing and GTM content positioning the UVP of our product/services portfolio to a senior audience.

If you are not already using MongoDB, let’s talk about what it can do for you and your organization. If you are, I’d like to hear how you are doing. You can find me at any contact point on the Connections page and I’ll be in touch soon.

Getting Engaged and Experienced in Vegas


Sitecore Symposium 2014 LV
WOW. What happens in Vegas: I’m writing about it, and analyzing it. Ok, before anyone goes scrambling for legal counsel and/or relationship counsel-ing, what I’m analyzing is Sitecore Symposium 2014 in Las Vegas: a slickly-produced conference with, I’m guessing, judging by the chock-full keynote hall, +/- 2,000 attendees. The event featured multiple tracks and sessions to choose from. While many vendor confabs feature pleas of “Go to the exhibit hall (we beg you) to visit our vendor sponsors who paid big bucks for very little traffic,” at Sitecore Symposium, the vendor partner tables and demos were set up right in the hallway connecting the breakout halls where many of the sessions were held. That was convenient and encouraged dialogue. Bravo, Sitecore. Others take note. I expect this to be a most-copied concept.


Judging by his razor-sharp, on-point keynote, Sitecore CEO Michael Seifert, who I interviewed on an earlier sojourn to Vegas, was born to the stage, personable and well-paced. The theme of Seifert’s talk, and of the overall conference, was “Own the Experience.” Another oft-repeated phrase at the event was “Customers for life.” In fact, Sitecore Chief Strategy Officer Darren Guarnaccia said in his Product keynote, “We want to be the preferred tool for customer experience developers, marketers and merchandisers…this is a platform for life.” In this analyst’s view, Sitecore is well on the way to actualizing that vision.


Some notable quotes and concepts jumped out of Seifert’s keynote:

“This is the opportunity of our times: to present the right experience at the right time… the most important opportunity in today’s environment.”
JeffTake: This is why both customer experience analytics and real-time analytics, two areas of focus for us at Stratecast, are so important today.

“An experience creates a lasting memory—and that beats content…but you must know the person to provide the experience, so that requires context…and you must have creative, emotionally-engaging content.”
JeffTake: we’re talking a lot about content and context, plus sentiment analysis, at Stratecast these days, and this encapsulates a lot of the ‘why.’

“Consumers are getting MAD: marketing attention deficit.” He explained they are bombarded with email, then Internet banners and popups, and they tune out whatever is perceived as an interruption or not relevant.
JeffTake: this gets to the essence of speaking with and making friends with your customers and prospects, not “marketing AT them,” which is supposed to be the hallmark of digital communications as opposed to so-called traditional marketing.


Seifert talked about how marketers now have plenty of tools at their fingertips, but that technology is starting to overwhelm them: they face too much complexity; it’s becoming an arms race. He showed this marketing tech chart and cited stats from the Digital Analytics Association that 50% of marketers’ time is spent gathering and analyzing data. And how, even after they’ve assembled all of that data, they still don’t get a comprehensive view of the customer, just bits of it.

Remember, Seifert pointed out, that Amazon, for example, is only selling online [JeffTake: at least for NOW], which makes it simpler, while other organizations have storefronts, distributors, other marketing, sales, and service channels and touchpoints from which they must gather info, and to which they must impart new ways of better serving customers. The idea is to capture all relevant data—but SIMPLIFY.


Seifert next brought Guarnaccia on stage, and Guarnaccia showed a few dashboard screens from the new Experience Profile. Throughout the event, a number of Sitecorians referred to the Experience Profile as “The exFile.”

Sitecore’s “exFile” (which, come to think of it, could also be a great name for a database of ex-spouses and other partners) provides a current and comprehensive view of the customer, based on data flowing into the also-brand-spanking-new Sitecore xDB (its Experience Database, built on MongoDB, with connectors and Sitecore application software). Guarnaccia explained how now you can collect all the information from social, internal, and external sources and really see who this customer is, what they like, don’t like, what they have been doing on your site (and other sites), and all in all, how they are interacting with your business. Thusly armed, you can predict what they might want to do or see next, and provide what he termed “experience optimization.” Which means to track, visualize, and predict, then generate, on the fly (meaning: in real time) dynamic personas and lists for immediate action.
JeffTake: first, this discussion points out the importance of Stratecast’s 39 Data Sources Enterprises Need to Access. Second, Sitecore nailed what Stratecast has been urging the market to do for years: provide solutions that empower users to ANALYZE THEN ACT.


Next Sitecore VP Product Marketing Mark Floisand joined Guarnaccia on stage, and the two discussed some of the platforms’ recent growth stages: version 7.0 was about content. 7.1 was mostly about the user interface (UI: the SPEAK framework). 7.2 was about integrating Commerce Server. With 7.5 (the current version as of Symposium), Sitecore re-architected the platform to add the new xDB. In 8.0, Sitecore 8, the company was adding exFile, plus easy test and optimization for all website changes plus integration with Cloud ML on Azure to do predictive re-segmentation and next-best-experience offers. They announced that Sitecore 8 was in so-called “lighthouse” customers now [JeffTake: heavy beta with a trusted shortlist of top customers] and would be available for technology previews in November, with GA slated for early 2015.

Next, Sitecore partner Coveo dashed up on stage with breaking news: a free version of its enterprise search product for all Sitecore customers (discussed in some detail here).

Commerce Server is an acquired addition and integration of e-Commerce capabilities (including merchandising, shopping cart, and more). Stratecast met separately with Sitecore Director of Product Marketing Wayne Smith, who told us they have been “skilling up” since the purchase in Nov 2013, and that a bunch of Commerce personnel came with the technology. Guarnaccia’s blog post talks about all of this and establishes a Customer Lifecycle that is far more extensive than most of what has passed for “product lifecycle” in the industry at large for, like, ever. With Commerce Server, Sitecore can now map, monitor, and act on everything from initial customer awareness through consumption and advocacy. JeffTake: this means acting as a reference customer, including on social media and as a positive contributor to things like a positive Net Promoter Score.

Author and marketing and sales strategist David Meerman Scott was an entertaining guest speaker, emphasizing, among other things, “humanizing” [personalizing] marketing. He also hit the real-time button HARD (a speaker after Stratecast’s real-time-analytics-espousing heart). Here’s a recap by Sitecore VP Marketing North America James Smith.


Microsoft has named Sitecore its top ISV partner two years running. Sitecore VP Business Development Jean-Paul Gomes told Stratecast “the Microsoft connection” pulls through major revenues for Sitecore. Gomes is an ex-MSFTer who is still well-connected there and spends much of his time at Microsoft’s Redmond, WA (Seattle-area) campus. Sitecore’s partner roster reads somewhat like the Library of Congress, but the company told Stratecast it is moving toward having fewer partners to focus on bigger targets. That sound like “we’re going up-market” to me.

Like virtually every provider, Sitecore strives for replicable processes, but the fact remains that most Sitecore implementations are still one-off affairs, because organizational structures, politics, digital maturity, and technical challenges are always different. Sitecore’s SBOS (Business Optimization Services) offering helps with this, and Sitecore has a number of implementation partners it certifies for various competencies.

Sitecore also has Komfo (social media monitoring, analytics and publishing) and Print Experience Manager (formerly Advanced Print Studio, or APS), but did not heavily emphasize these at the conference. Nor was there talk about privacy and security in any of the sessions or briefings we attended. Best insight on privacy was an answer from panelist Avanade: “Get tight with your legal team.” (Avanade is a Microsoft/Accenture partnership that provides consulting, implementation management, and managed services.) We at Stratecast have a LOT to say about privacy…but hey, that’s part of why we are here.

From partnering to competing, Sitecore sees its primary competitors as Adobe and Salesforce—to which Stratecast would add others, principally Marketo, Silverpop, and IBM Tealeaf, and on the telecom side, Alcatel-Lucent, Comptel, and Amdocs Actix.


Ok, as the cynical analyst, I keep wondering how Sitecore can continually remake and reposition itself to meet changing markets:

In years past, the biggest name in chargeable web content management (WCM, which, btw, is still raging full force under the hood)…
‘Add analytics and marketing automation’ and it’s a customer engagement platform…
Now a MongoDB-fueled customer experience solution.

Somehow Sitecore does it. Or is doing it. Again. Remember, GA of Sitecore 8 is not until sometime in 2015. So the jury is still out, the pudding is not really proof-ready, etc., etc.—we’ll see what we see when Sitecore 8 hits GA-ness next year.

Many Sitecore customers have are running multiple instances of the platform and using different aspects of it, depending on their varying levels of digital maturity. Additional functions are being added and carefully integrated, because Sitecore’s most advanced customers want them and Sitecore believes all customers will need them.

Event-wise, Sitecore Symposium 2014 was really well managed. Oh, and each attendee got a copy of the book Connect: How to Use Data and Experience Marketing to Create Lifetime Customers, written by three Sitecorians: Lars Birkholm Petersen, Ron Person, and Christopher Nash.

In fact, hey wait a minute, this analyst provided a quote for the book and I’m still waiting for my copy signed by the authors, or at least by Petersen. How about it, Sitecore?