Your data needs to keep up with the state of play today. Event-driven architecture will get you up to speed

Gallery

The Pop-Up Playground Party at AWS re:Invent 2019, sponsored by MongoDB and Confluent, was an exclusive, interactive event featuring immersive art, light, gaming, and play activations with an open bar, DJs, and more, at the Industrial Las Vegas.

Attendees got to create unforgettable memories (or at least killer Instagram photos) at the glowing ball pit, DIY graffiti walls, retro arcade games, and other immersive activities, such as getting a temporary tattoo in a sports car. Meanwhile, the Hood Internet and Vegas’s hottest DJs kept the soundtrack fresh all night long!

In a word, the Pop-Up Playground was: fun.

Something that is way less fun: being one of the many companies that increasingly find it impossible to keep up with the pace of business because their data platforms can’t keep up. Under those circumstances life is anything but a party, in fact, the opposite of a party. 

That’s why MongoDB and Confluent teamed up, not just to throw the best party at re:Invent but to help companies run their businesses in real time. 

An effective approach to real time could not come at a better time, because many companies are still trying to meet their data needs in ways that don’t match modern business reality. Some are mired in 40-year-old relational database technology. Others think updating to a NoSQL database is positioning them for the future. They soon learn that while getting data in may be easy enough, getting insights out is a whole different story, and what they are actually positioned for now are niche applications, not an effective new data strategy.

The results of pursuing the wrong data strategy can be harmful or fatal. CNBC Markets found that the average lifespan of S&P 500 companies has fallen from nearly 60 years in the 1950s to less than 20 years today. Innosight modeled the rate of attrition from the S&P 500 and predicted that over the next 10 years, half the companies will be replaced: drop out, go out of business, or be acquired, as digital transformation lowers barriers to entry for new players and helps drive out incumbents trapped in legacy infrastructures and processes.

The traditional request-driven data architecture that still exists at many companies helps keep them trapped in legacy limitations. It requires users and applications to make requests and wait until the requested information becomes available. Waiting for data kills opportunity and agility.

By contrast, event-driven architecture, which I posted about on jeffcotrupe.com,  proactively makes a stream of data from source systems (producers) available in real time. Consuming applications and services (consumers) subscribe to topics of interest and consume data at their own pace. Capturing and acting on events in real time enables systems to react automatically and immediately to events. This helps a company rapidly position itself to outflank competitors. Detecting operational errors lets it take immediate corrective action. These benefits translate into not only operational excellence and cost savings but also enhanced customer experience as the company optimizes customer-facing processes. More broadly, an event-driven architecture helps the organization improve business agility.

The key driver today in making the need for real-time data an organizational imperative is the emergence of microservices. Microservices architecture breaks up monolithic applications into small, discrete services or functions. It creates self-sufficient sprint teams empowered to bring new capabilities online independently of each other, then over time evolve and upgrade their microservice without impacting adjacent microservices. That is the essence of agility.

As beneficial as microservices can be, though, they require the ability to work with large volumes of data that change frequently, which is a challenge many existing systems cannot meet. Trying to implement microservices in a legacy relational database incurs the pain and friction of having to define a schema in the database and re-implement that same schema again to effect object-relational mapping (ORM) at the application layer. Then your development team has to repeat the process, first for each microservice and then every change to the data model as application functionality evolves.

With MongoDB, data modeling for microservices is easy, which is a big reason MongoDB is at the core of many event-driven systems today. MongoDB’s flexible document model gives you the best way to work with data, lets you intelligently place data where you need it (and when, as in immediately), and gives you the freedom to run anywhere. MongoDB helps you move at the speed your users demand. It gives you the power to launch new digital initiatives and bring modernized applications to market faster, running reliably and securely at scale, unlocking insights and intelligence ahead of your competitors.

 

By starting with the core MongoDB data platform and binding in complementary technologies, MongoDB provides the data persistence heart of an event-driven architecture. MongoDB and Confluent work together to enable you to readily build microservices and event-driven architectures to become an agile organization.

Confluent Platform, including Apache® Kafka® and Kafka Connect, is designed as an event messaging queue for massive streams of data that sequentially writes events into commit logs, allowing real-time data movement between your services and data sources. The MongoDB Connector for Apache® Kafka® — developed and supported by MongoDB engineers, and verified by Confluent as a first-class component of Confluent Platform — simplifies building robust, reactive pipelines to move events between systems. You can use MongoDB as a sink (consumer) to ingest events from Kafka topics directly into MongoDB collections, exposing the data to your services for efficient querying, enrichment, and analytics, as well as for long-term storage. You can also use MongoDB as a source (producer) for Kafka topics; in this mode, data is captured via Change Streams within the MongoDB cluster and published straight into Kafka topics. These capabilities enable consuming apps to react to data changes in real time.

MongoDB-powered event-driven architectures are at work in a range of user cases including IoT and other time series applications; financial services; AI; predictive maintenance, primarily in manufacturing but also in other verticals; Web activity tracking and log aggregation; and as an operational data layer (ODL) integrating and organizing siloed enterprise data to make it available to all users and consuming apps. Customers who have deployed event-driven architecture powered by MongoDB include Ticketek, EG, ao.com, Man AHL, and comparethemarket.com

The figure below shows MongoDB and Confluent working together in an event-driven architecture supporting a microservices-based e-commerce application.

In this scenario, fuel costs to ship some items have just gone up, which could impact pricing. This produces events about the cost increase and places them into Apache Kafka. The Pricing microservice consumes the event, analyzes it against existing data, and produces events conveying the new pricing. MongoDB Atlas captures this data and, through the MongoDB Connector for Apache Kafka, publishes it into Kafka topics, which makes the data available to all consumers. Microservices directly impacted by pricing changes, such as those that manage inventory, marketing, promotions & coupons, point of sale (POS), and the e-commerce provider’s order management system (OMS), consume the price change events and update their individual databases accordingly. MongoDB Atlas aggregates and persists data from all microservices, enriches event streams with data from other sources, including historical data, and provides a central repository. This enables applications and users to benefit from all data across all microservices and provides a unified view of state across the e-commerce provider’s enterprise.

To learn more, download our white paper, How MongoDB Enables Real-Time Data with Event-Driven Architecture, and visit our Real-Time Analytics page.

Show me some skin(care)

Gallery

My younger daughter is offering the best line of affordable skincare products on the market. One of their many features that doesn’t come with most other skincare products is a hassle-free 60-day money-back guarantee. They’re for women AND men, so “as beautiful as my skin is” =;-D I am using them myself. Click the little FB logo below, or click here, to enjoy the video and learn more.

Never Mind the Carnivores, Here’s the Vegans

Gallery

To me, “going vegan” is not about being trendy or better-than-thou, showing anyone up, or guilting anyone into doing what I have come to believe is the right thing. It’s just what my wife and I have done in our household after our younger daughter introduced us to it two years ago. I’m thrilled to see this new nutritional path increasingly going mainstream, including stories in major media venues.

I also laugh or cringe at some media “findings” about plant-based nutrition.

One editor wrote about quick-eats-and-treats for Super Bowl parties. To me his one token vegan entry was unappetizing in print and after trying it, his taste buds agreed. So, to him, that was it. Case closed. Settled science. Forget anything vegan for your next party! You and your guests should keep chowing down on hamburgers and wings and hot dogs, and pizzas drowning in meat. Jam as much animal product down your throats as possible.

Not that I am bitter.

Another writer dug deep to find out that one of the popular new plant-based burgers had, she said, “similar levels of sodium to a regular beef hamburger.”

Media aside, I also marvel at some of my non-media fellow humans who stop wolfing down steroid-laden, processed meats, Cheez-its, and Mountain Dew just long enough to shoot down positive vegan stories with scattered concerns and cheap shots, such as:

Q: “If you vegans don’t want to eat meat, why do you want something that tastes like meat?”
A: Um…why would Tesla and others make EVs that drive like gas-powered cars (or way better)?

No one ever said they didn’t like the TASTE of meat and dairy products. We’re eating differently for three main reasons: [1] We are no longer willing to contribute to the demand for meat and dairy that causes “humans” to torture and kill animals for food.|e| [2] We want no part of the human health issues around eating red and processed meat, which is associated with heart disease, diabetes, and cancer. [3] We will no longer contribute to the demand for meat that accelerates climate change and drains natural resources to raise livestock for food.|f||g|

Point: “These plant-based foods are too processed.”
 Counterpoint: As laid out here, and paraphrased in this post, Beyond Meat and the Impossible Burger have better nutritional profiles than beef burgers: fewer calories, more protein, and less fat.|h|

Compared to a four-ounce beef burger with 20% fat content, a Beyond Burger has 20 fewer calories, three fewer grams of fat, and one MORE gram of protein. An Impossible Burger has 50 fewer calories, eight fewer grams of fat, and the same amount of protein. Both plant-based burgers have zero cholesterol, compared to 80 milligrams in a beef burger. And both have more fiber, another essential to good health.|i|

Statement: “You can’t get protein unless you eat meat.”
 Fact Check: WRONG.

We get plenty of protein every day at every meal in things like soy milk, oatmeal, nuts, chia seeds, tofu, beans, chickpeas, nutritional yeast, whole-grain breads, green peas, quinoa, wild rice, and high-protein fruits and vegetables. One thing we don’t get is vitamin B12, and we take that as a supplement. |j|

Often Heard Locally: “I’d like to try vegan, but my husband’s a meat-and-potatoes man and he works hard and he needs his meat.”
Speaking Truth to Carnivores: a growing number of the world’s top athletes in the most demanding physical sports are vegan, so…whatever your job may be, I’ll bet it can work for you.

Who are they? NFL players including DeAndre Hopkins, Malcolm Jenkins, Matthew Stafford, and Marcus Mariota. NBA players including nine-time NBA All-Star Chris Paul, Damian Lillard, Kyrie Irving, Wilson Chandler, Al Jefferson, Garrett Temple, Enes Kanter, JaVale McGee, and Jahlil Okafor. Pro soccer players in the British Premier and other leagues including Jermain Defoe, Chris Smalling, Fabian Delph, and Jack Wilshere. World-class athletes across the most demanding physical sports such as tennis icons Novak Djokovic and Venus Williams, boxer David Haye, UFC mixed martial arts fighters Nate Diaz and Abel “Killa” Trujillo, bodybuilder Barny du Plessis, snowboarder Hannah Teter, skater Meagan Duhamel, Formula 1 champion Lewis Hamilton, ultramarathoner Scott Jurek, and more. NFL living legends Tom Brady and Aaron Rodgers are not fully vegan, but their diets are reportedly 80% plant-based.|k|l|m|n|o|

Continue reading

Mac Links LIVE!

Gallery

Ha, “I’ll explain.”

For I think 10-15 years now I’ve been using the Office suite on PC to make PDFs of Word, Powerpoint, and Excel files and never gave links a second thought. You embed links in text like this in the source file and they’re live in the PDF. Fonts and layout are identical to what you created in the source file.

Not on Mac.

Oh, if you simply type out links as text, e.g., https://www.crunchbase.com/person/jeff-cotrupe, they do save as live links in PDFs. But it is self-evident why that is sub-optimal. Pretty much everywhere in all applications people embed links in all content like this, and that is the standard.

Create a file in Office on the Mac, save as PDF and you’re presented with two choices:

  • Saving your PDF as “Best for printing” preserves fonts/layouts but kills the links.
  • Saving as “Best for electronic and accessibility (uses Microsoft online service)” gives you live links, but is also dependent upon which fonts Microsoft has loaded on those servers. In any file I’ve ever tested, this method massacres fonts and layouts and is unworkable.

After much experimentation and testing, here are the only two ways you can use Word on a Mac to create PDFs with embedded live links AND the fonts and layout of your choice:

[1] Use Word for Mac to create your doc. Save as Best Print Quality. Your embedded links are dead. Buy Acrobat Pro DC for $450 or license it for a year for $180. Open your PDF and the source Wordfile. Copy link locations for all links in Wordfile and paste them in into new link boxes you create over those same words or phrases in PDF.

[2] Buy Parallels, Windows 10, and Office for your Mac. Parallels virtual machine frees Word to do on Mac what it does on PC: save embedded live links in PDF. Not sure what it cost my company to buy Parallels and Windows 10 licenses, and the new separate license for Office to use in Parallels. 

Long way and lot of money to go just to create live links AND not demolish your fonts and layouts when saving Word to PDF on a Mac, right? But those are your choices.

I bet Microsoft, Apple, and Adobe could get together and fix this in a New York (or Seattle, or Silicon Valley) minute. But there’s a lot of money changing hands in #s 1 and 2 above, so…

I’m now at MongoDB!

Gallery


I am now a proud member of the Product Marketing team at MongoDB: the first database company to go public in more than two decades (NASDAQ: $MDB), its business growing at ~50% YoY, with the technology and vision to take on the multibillion-dollar incumbents as it disrupts and reshapes an entire industry. As an analyst I had forecast a total market opportunity of $67.89 billion in big data and analytics by 2019, growing to nearly $111 billion by 2022. I am excited to be at a truly global company capturing a sizable and growing share of that opportunity!

My role is Senior Solutions Marketing Manager, a core part of an energetic and globally distributed team reporting to the Senior Director, Products and Solutions, based in the UK. I am responsible for driving solutions marketing and GTM content positioning the UVP of our product/services portfolio to a senior audience.

If you are not already using MongoDB, let’s talk about what it can do for you and your organization. If you are, I’d like to hear how you are doing. You can find me at any contact point on the Connections page and I’ll be in touch soon.

Your two most pressing Outlook iCloud Sync issues SOLVED

Gallery

If you are reading this, you probably want to be able to add or change contacts or calendar items in Outlook on your computer and have them reflected, ‘through the magic of iCloud,’ on your iPhone, and vice versa, from phone to computer. That was all I wanted, but I hit some walls that seemed insurmountable until I found solutions posted by some expert and helpful users. I decided to post them together to offer you, the reader, one-stop shopping to these fantastic fixes.

ISSUE: “Setup cannot continue because Outlook is not configured to have a default profile.”
This one smacks you in the face right out of the gate, after you have installed iCloud and are in setup. You check the box to tell iCloud to sync Mail. Contacts, Calendars, and Tasks, hit apply–and you get the message above. After rapidly descending on an estimated 11 zillion sites with solutions that did not work, I found one that did.
SOLUTION: Close iCloud. Open File Explorer, go to c: \ Program files(x86) \ Common files \ Apple \ Internet and find the file simply called iCloud (it’s icloud.exe, but the extension may not be visible). Right-click it and choose Properties. Click the Compatibility tab and you should see something like this:

The compability troubleshooter button is highlighted, and I pressed that and put the system through its paces, but ultimately all you have to do is check the box and select the pulldown menu so it reads as it does here: “Run the program in compatibility mode for Windows 8.” Press [OK] at the bottom.

Now open iCloud again, check the box for Mail. Contacts, Calendars, and Tasks, hit apply, and it should go through its sequence to set up iCloud and Outlook to sync.

ISSUE: Contacts are syncing but calendar items are not.
I again found a number of sites with well-meaning solutions that weren’t, but thankfully found one pretty quickly that worked.
SOLUTION: Open Outlook, choose calendar, and double-click in a date box to set up a new Appointment. You’ll notice Calendar at top left.

Your calendar may be defaulting to Outlook Data File as mine did:

If so, use the slider to page down, find your iCloud calendar, and check the box for Calendar:

I was going to say this is not a permanent solution because you will have to do this every time. However, it appears that if I just remember to keep the iCloud calendar checked instead of letting Outlook select Outlook Data File as the default calendar to show, new calendar invites I create are coming up correctly in iCloud calendar.

To close the loop, I double-checked whether calendar entries I create in iCloud are syncing to Outlook. iOS lets you preset the calendar in which new Appointments are created. So yes, new Appointments I create on my phone are syncing fine in Outlook without any adjustment.

As some of my friends in other parts of the world like to say: “There you are. All sorted.”

I am guilty of having saved content and screenshots in my files, but not links to the pages where I found this sage advice, to give credit where due. Thank you, solution providers. You know who you are, and you have my eternal gratitude.

 

WELCOME to a more competitive e-commerce market

Gallery

Attended the eTail West event two years running…placed two online orders during roughly the same period with a company I lovingly dub the Retail Death Star…had a deadline to meet for the IDG Infoworld blog…and it led to one of my most fun-to-write blog posts ever. The WELCOME mat above plays a pivotal role in the piece, but IDG wouldn’t run this photo because it doesn’t publish anything but its own internally approved pix. So here it is in all its boldly leaf-strewn glory, beckoning you to walk right over it and enjoy all the e-commerce action here.

Before I go: some of the arboreal wonders giving the mat that leafy look.

Innovating for Dollars: AI / ML / Deep Learning / Cognitive for Financial Institutions

Gallery


When Letterman was still hosting the Late Show, from time to time he’d say: “And if all THAT weren’t enough…and by golly, don’t you think it ought to be,” and would go on to mention something else that was noteworthy about the show or a guest.

Bear with me, I’ll go back to that in a moment, and it will tie into our discussion…which is about AI. AI is forecast to drive GDP gains of $15.7 trillion globally by 2030.* No organization can afford to miss out on its share of those trillions in opportunity. Adoption is low thus far, however. In Stratecast’s 2017 Big Data and Analytics Survey, only 20% of organizations said they are considering, planning, implementing, or using it. We think one part of it is market confusion; another is organizations not seeing the link between AI and revenue—both growth and retention.

Let’s first dispel some confusion surrounding AI. My practice analyzes big data and analytics (BDA), and here is where AI fits at a functional level: the Business Process and Strategic Analytics (BPSA) area of the BDA market. BPSA represents about one-quarter of the overall BDA market, which we assessed at more than $53 billion in 2017, and which we forecast to grow to nearly $68 billion by 2019.

Now let’s talk about other linkages between AI and money. Ever heard of Alexa and Siri? Amazon? Facebook, Google, Netflix, Tesla, and Uber? These and a multitude of others are making money right now by applying AI to their businesses. But wait, you say, what about financial institutions? Well, how about one of the largest online financial trading services in the world? It has used AI to reduce support costs by 80%. Or consider how, with hundreds of forms and more than 350 online apps, associates at a top-10 Wall Street investment firm were spending an average of 20–30 minutes looking for each form or app. The firm is now saving an estimated $32 million annually by applying AI to those and related processes. Even these examples, however, reflect a financial services market that is only beginning to scratch the surface of all the ways AI can benefit financial institutions. Customer-facing applications of AI include learning customer patterns and motivations to help guide them toward better financial decisions. In the back office, AI can, similarly, guide a financial institution’s own investment decisions. AI can automate tasks in many areas including underwriting, reconciliation, the development of risk models, and basic handling of incoming data and queries.

Sounds good, right? But delivering on the promise of AI requires a vision for applying smart analytics to the business. Nowhere is that concept more fitting than when talking about Tableau. Tableau provides the foundation underpinning the adoption of new and emerging technologies with an enterprise platform that covers all the bases in governance and security to help financial service companies guard against security breaches and ensure privacy compliance. Tableau offers rapid performance against massive datasets, an effect now accelerated by Hyper, its fast main-memory database system designed for simultaneous OLTP and OLAP processing (transactions and analysis in a single system) without compromising performance. Powerful, self-service analytics drive innovation, encouraging employees to discover opportunities for new products and services, contributing to customer and revenue growth—and enabling them to quickly run scenarios to assess the impacts of new business models such as blockchain.

Back to Letterman: “And if all THAT weren’t enough…and by golly, don’t you think it ought to be,” Tableau also has a vision for smart analytics that transcends AI-supporting BDA firepower with some pretty impressive AI building blocks. On the NL front, Tableau acquired Cleargraph and is combining Cleargraph’s NLP capabilities with Tableau’s existing Eviza natural language interface; and the company has partnerships with Automated Insights and Narrative Science to add NLG capabilities to the mix. In development are Tableau’s new Recommendations Engine, which will enable discovery, help users reuse the work of others, and leverage knowledge of their communities; Model Automation, which will offer smart defaults, saving time and providing ease of use; and Automated Discovery, to help customers discover hidden insights and answer more complex questions.

Planning on revenue growth? Failure to harness the power of AI could be a showstopper. Tableau has the content and the connections to ensure that the show will go on.

*PwC, AI to drive GDP gains of $15.7 trillion with productivity, personalization improvements, available here

How iPhone Got Her Outlook Back

Gallery

After working fine for years, suddenly iPhone and Outlook stopped syncing contacts and calendars. Outlook started throwing off messages a la, “Outlook had a serious problem with the iCloud Add-In, do you want to disable it?,” and the iCloud tab and Refresh button disappeared from Outlook. None of what I’ll politely call the ‘easy fixes based on standard instructions’ you so often see about things like this all over the Web did a thing to change the situation. Here’s how I solved it:

[1] In Outlook 2010: File > Options > Add-ins. That brought me here:

This is actually the ‘after’ view–after I fixed this. The view when I attacked this problem was identical except that iCloud Outlook Addin was appearing under Disabled Application Add-ins. No, I am not disabling it and going through the process all over again for the sake of this piece. There is journalistic integrity–and there is “Why in God’s name did you touch it again after you rescued it the first time?!” But I digress…

Anyway, noticing the file location paths, I typed in a separate note the file location for the iCloud Add-in shown above: C: > Program Files (x86) > Common Files > Apple > Internet Services > APLZOD32.dll. You’ll see why in a minute.

The page is preset to Manage COM Add-ins, and I pressed [Go].

[2] That brought me here:

In this view I selected the Cloud Outlook Addin* to highlight it, as shown above, and pressed [Remove].

[3] Next, in this same view I next pressed [Add…], and navigated to the file location I had found above, which brought me here:

I selected the file, saw it was now listed in Active Application Add-ins, and noticed that, miracle of miracles, the [iCloud] tab at the top of the Outlook window, and the [Refresh] button, had returned. Yes, syncing iPhone to Outlook could actually be a refreshing experience again.
So far so good, but any dev and probably many readers know what comes next: TESTING.

I held my breath as I tested it: added and deleted Calendar and Contact items, and modified existing items, on iPhone and in Outlook, and they are now updating both ways—iPhone-to-Outlook and Outlook-to-iPhone—either immediately in front of my eyes, or when I hit Refresh.

“Victory is mine”…and, if you’ve been seeking a solution to this problem: YOURS. We’re all in this together.

*Whoever was in charge of placing dashes in filenames missed on the iCloud and VBA (Microsoft Visual Basic for Applications) “addins,” or as they’re widely known: Add-ins.

Kaley and me: breathless anticipation

Gallery

Kaley-Cuoco-Undergoes-Sinus-Surgery-665x385Hello again, sports fans. It’s been forever (since November 2014) that I posted anything here, and for those of you who may still be following this blog, I appreciate your patience.

You’re probably wondering about the gratuitous use of a photo of the gorgeous and personable Kaley Cuoco at the top of this post. Well, first of all, the picture (and tag) may just get some of you to read my ramblings. Well, that and the “breathless anticipation” headline. More importantly, though, my chance viewing of Kaley in a brief snippet of a TV program I rarely get to watch, the Ellen DeGeneres Show, ultimately led to me making a beneficial change in my health and life.

Seeing Kaley on an edition of the Ellen show featured here, I realized I had left at least one stone unturned in my quest for better breathing. After dialogues with various physicians, and a seemingly endless series of medical visits and tests and questions and questionnaires, I finally, in early June, had the surgery Kaley had talked about on Ellen: a surgical procedure to correct a deviated septum. In recovery, even through the anesthetic haze, I already felt better. Weeks later, although I’m still sniffly and finding out that “allergies don’t take a holiday for deviated septum surgery,” I’m breathing better and snoring is for the most part gone.

Doctor, will I be able to play the piano after surgery? Will I look as gorgeous as Kaley (in a male way, to maintain decorum and also keep my Man Card) after surgery?

Dr. Robert Jacobs, who did the surgery, and our family doctor, Dr. Jeff Newman, did not ask for plugs in this post and do not even know I’m blogging about this, but Dr. Jacobs and ENT Associates did a great job for me, and Dr. Newman and Cardinal Medical Center recommended Dr. Jacobs for this surgery. I enthusiastically endorse them both.