We’re getting Big Data all wrong, and it’s holding us back. By making a fetish of the volume of data we’re collecting, we’ve completely overlooked the most important aspect of our data: analyzing it.
Such analysis is often assumed to be the province of data scientists, those magical unicorns that take one look at a company’s data and declare, “Buy low, sell high!”
Because data scientists can be the difference between success and failure in a company’s use of its data, finding the right kind is critical. It turns out that discovering the right data scientist is similar to analyzing one’s data: you need to make sure you’re hiring the right kind, and that you ask them the right questions.
Big data technologies have evolved at a torrid pace that shows every sign of continuing in 2015. MapR CEO and co-founder John Schroeder predicts five major developments will dominate big data technology in the new year.
n just a few short years, big data technologies have gone from the realm of hype to one of the core disruptors of the new digital age. 2014 saw big data initiatives inside the enterprise increasingly move from test to production. In 2015, big data will push further into the enterprise with even more use cases — specifically real-time use cases — says John Schroeder, CEO and co-founder of Hadoopdistribution specialist MapR.
“This is the year that organizations move big data deployments beyond initial batch implementations and into real time,” Schroeder says. “This will be driven by the realization of the huge strides that existing industry leaders and soon-to-be new leaders have already made by incorporating new big data platforms into their analytics with “in-flight” data to impact business as it happens.” Schroeder says five major developments will dominate 2015.
1. Data Agility Emerges as a Top Focus
Data agility has been one of the big drivers behind the development of big data technologies, as the processes around legacy databases and data warehouses have proven too slow and inflexible for many business needs. In 2015, Schroeder says data agility will become even more central as organization shift their focus from simply capturing and managing data to actively using it.
“Legacy databases and date warehouses are so expensive that DBA resources are required to flatten summarize and fully structure the data,” he says. “Upfront DBA costs delay access to new data sources and the rigid structure is very difficult to alter over time. The net result is that legacy databases are not agile enough to meet the needs of most organizations today.”
“Initial big data projects focused on the storage of target data sources,” he adds. “Rather than focus on how much data is being managed, organizations will move their focus to measuring data agility. How does the ability to process and analyze data impact operations? How quickly can they adjust and respond to changes in customer preferences, market conditions, competitive actions and the status of operations? These questions will direct the investment and scope of big data projects in 2015.”
2. Organizations Move from Data Lakes to Processing Data Platforms
In some ways, 2014 was the year of the data lake (or data hub), an object-based storage repository that stores raw data in its native format — whether structured, unstructured or semi-structured — until it’s ready for use. Data lakes have a strong value proposition in that they represent a scalable infrastructure that’s economically attractive (with a reduced per-terabyte cost) and extremely agile.
Schroeder says that the data lake will continue to evolve in 2015 with the capability to bring multiple compute and execution engines to the data lake to process the data in-place. That’s not only more efficient, it creates a single point of governance and a single point of security.
The IDG TechNetwork has expanded its global online advertising network into China, following its parent company International Data Group (IDG), which has supported the market for 34 years.
The launch of IDG TechNetwork China, announced Monday, means that IDG now supports data-driven marketing and premium programmatic buying to more than 60 Chinese language magazines, newspapers and Web sites.
Peter Longo, CEO of U.S. Media for IDG Communications, believes the move makes the company the first global technology-focused ad network to enter the Chinese market. The network supports more than 570 publisher sites and reaches more than 130 million tech enthusiasts, enterprise tech buyers and gamers.
Based in Beijing, IDG TechNetwork China is a fully owned business unit of IDG China led by CEO William Xu.
For brands, the move means an easier transition into China to reach the local market and better access to premium inventory they can purchase and optimize through one media group, the IDG TechNetwork China.
Marketers will have access to auctions and private marketplaces, data management platform and demand side platform services, along with increased targeting capabilities and multiple ways of acquiring inventory through direct placements and exchange based trading, per Longo.
“They will be able to buy with confidence against premium inventory from IDG, a media company that has been doing business in China for over 30 years,” he said.
All owned-and-operated Web sites in China become part of the IDG TechNetwork China along with selected premium partners that we have chosen to work with us. This provides increased scale to offer their advertising partners, as well as the ability to do business with new partners, such as DSPs and agency trading desks.
The company will now collect first-party data for its network of Web sites, allowing for better performance and return on investments for advertisers.
One of the top goals for business leaders today is to better understand, engage with and retain their customers[i]. This involves making the most of the ever-growing volume of customer data available to build integrated, three-dimensional profiles of customers and to identify patterns and trends. Many firms turn to the roles closest to the customer to deliver this insight.
Recent research undertaken with PwC[ii] reveals that nearly two-thirds of European and just under half of North American mid-market firms believe their marketing teams have the best skills to extract insight from information, and around half (46% and 57% respectively) say the same for their customer service and insight teams.
Yet conversations with marketing leaders reveal that the teams in question are far less confident about their ability to achieve this.
One study[iii] found that a third of executives believe that being able to use data analytics to extract predictive findings from big data is the top skill required of their marketing professionals. However, just under half admit their own team lacks this skill. Another[iv] discovered that an overwhelming 82% of marketing leaders feel unprepared to deal with the data explosion, and only 59% say they have the skills to analyse and understand customer behaviour across all channels.
Despite this clearly recognised skills gap, only one in five marketing professionals is expected to receive formal training in data analysis this year[v].
In short, many firms could be passing data to teams that are ill-equipped to do it justice. Missing out on rich customer insight is just one of the risks. Our research found that marketing teams are increasingly given free access to sensitive and confidential customer information in order to extract intelligence, but are rarely held accountable for keeping it safe.
We discovered that less than one per cent of mid-market firms think teams such as marketing and customer insight should have a responsibility for information protection. Many (39%) place this responsibility firmly at the door of the IT security manager.
This is all the more worrying when you consider the fact that marketing departments are often at the forefront of flexible working practices[vi], allowing staff to work from home or while travelling – often without providing adequate guidance and support.
We found that one in three marketing professionals works from home two-to-four days a week, more than most other job roles. A third undertake confidential or sensitive work while travelling on public transport; one in four throw documents into insecure bins away from the office – and 48% send or receive work documents over a personal email account, at times using an insecure wireless network (12%). However, just a third of the employers surveyed provide secure remote intranet access for marketing professionals working from home, or offer guidelines or policies on how to handle sensitive information.
Within the next five years, more than 90% of all IoT data will be hosted on service provider platforms as cloud computing reduces the complexity of supporting IoT “Data Blending”
FRAMINGHAM, Mass., December 3, 2014 – International Data Corporation (IDC) today hosted the IDC FutureScape: Worldwide Internet of Things 2015 Predictions Web conference. The presentation provided organizations with insight and perspective on long-term industry trends along with new themes that may be on the horizon. The Predictions Web conference series and accompanying IDC FutureScape reports are designed to help company leaders capitalize on emerging market opportunities and plan for future growth. An audio replay of today’s Web conference will be available this afternoon. To access the replay, please visit: http://bit.ly/IDCioTFutureScape2015.
ClicktoTweet: @IDC Reveals #Worldwideinternetofthings #Predictions2015 – Register for the webcast replay here http://bit.ly/IDCioTFutureScape2015
The predictions from the IDC FutureScape for Internet of Things are:
IoT and the Cloud. Within the next five years, more than 90% of all IoT data will be hosted on service provider platforms as cloud computing reduces the complexity of supporting IoT “Data Blending”.
IoT and security. Within two years, 90% of all IT networks will have an IoT-based security breach, although many will be considered “inconveniences.” Chief Information Security Officers (CISOs) will be forced to adopt new IoT policies.
IoT at the edge. By 2018, 40% of IoT-created data will be stored, processed, analyzed, and acted upon close to, or at the edge, of the network.
IoT and network capacity. Within three years, 50% of IT networks will transition from having excess capacity to handle the additional IoT devices to being network constrained with nearly 10% of sites being overwhelmed.
IoT and non-traditional infrastructure. By 2017, 90% of datacenter and enterprise systems management will rapidly adopt new business models to manage non-traditional infrastructure and BYOD device categories.
IoT and vertical diversification. Today, over 50% of IoT activity is centered in manufacturing, transportation, smart city, and consumer applications, but within five years all industries will have rolled out IoT initiatives.
IoT and the Smart City. Competing to build innovative and sustainable smart cities, local government will represent more than 25% of all government external spending to deploy, manage, and realize the business value of the IoT by 2018.
IoT and embedded systems. By 2018, 60% of IT solutions originally developed as proprietary, closed-industry solutions will become open-sourced allowing a rush of vertical-driven IoT markets to form.
IoT and wearables. Within five years, 40% of wearables will have evolved into a viable consumer mass market alternative to smartphones.
IoT and millennials. By 2018, 16% of the population will be Millennials and will be accelerating IoT adoption due to their reality of living in a connected world.
“The Internet of Things will give IT managers a lot to think about,” said Vernon Turner, Senior Vice President of Research. “Enterprises will have to address every IT discipline to effectively balance the deluge of data from devices that are connected to the corporate network. In addition, IoT will drive tough organizational structure changes in companies to allow innovation to be transparent to everyone, while creating new competitive business models and products.”
The IDC FutureScape report that this Web conference is based on will be published and available within the next 24 hours. To learn more about IDC Predictions and IDC FutureScapes, please visit:www.idc.com/Predictions2015.
IDC envisions 2015 will be a year of accelerating innovation on the 3rd Platform
Jaideep Mehta, Managing Director, IDC South Asia says, “It has been about two years since organizations started to explore the benefits 3rd Platform technologies could unlock for them. 2015 is finally expected to be a year of widespread adoption of the 3rd Platform – Social, Mobile, Cloud and Big Data. IDC expects the businesses to move from a saturated 2nd Platform to a now thriving 3rd Platform era. Recovering markets, growing capabilities and innovating leaders will act as catalyst during this transition phase making 2015 a significantly positive year for the India IT eco-system.”
IDC revised IT spending growth in the Asia/Pacific (excluding Japan) (APeJ) region down from 8.7% to 5.8% in 2014 followed by a very modest increase to 6.0% in 2015. IT spending growth for the rest of the 2014-2018 forecast period is expected to climb upwards to 6.4% in 2017.
IDC expects the APeJ region to remain a most reliable engine for growth with multinational companies (MNCs) and Asian enterprises alike continuing to relentlessly look to Asia for future opportunities.
More insights will be revealed in a forthcoming report, “IDC Asia/Pacific (excluding Japan) ICT 2015 Top 10 Predictions”.
Drawing from the latest IDC research and internal brainstorming sessions amongst IDC’s regional and country analysts, the following are the top 10 key ICT predictions in 2015 that IDC believes will have the biggest commercial impact on the APeJ ICT market.
1) US$15 billion of government funding in 2015 will turn ICT plans into battlefields innovators
In 2015, IDC expects government ICT investments to be focused on the consolidation and streamlining of scarce ICT resources, the attainment of better management tools for effective decision making, and cyber-security.
In the next two to three years, IDC expects several regional authorities to utilize new sourcing models for transformational ICT, such as 3rd Platform technologies (i.e. cloud, Big Data/ analytics, mobility and social), continued Smart City programs, connected smart machines and intelligent sensors (i.e. edge computing), and the Internet of Things (IoT).
2) 60% of enterprises in 2015 will structure IT into core vs Lines of Business (LoB) IT
In 2015, IDC predicts that 60% of enterprises will structure their IT departments into two functional groups: Core IT and a separate LoB IT function. For larger organizations, these groups will become physically distinct entities, but for most Asia/Pacific enterprises this separation will be logical, as the two kinds of roles will be distinctly different but the reporting structure may not differ.
3) The software-defined battle lines will get defined in 2015
The hybrid cloud, or federated datacenter, is still the current architecture of choice for organizations trying to align their IT infrastructure to the demands of the business.
Looking ahead to 2015 and based on the IDC Asia/Pacific Transformative Infrastructure (TI) Index, between 20-25% of all organizations will already have adopted Software-Defined Networking (SDN), Software-Defined Storage (SDS), or Software-Defined Datacenter (SDDC) to deliver on their hybrid cloud architecture (such as automation, showback and service catalog capabilities) across the region.
4) The agile development team will be in high demand in 2015 with growth in DevOps adoptions
IDC’s IT Services Survey found that 45% of businesses are undergoing or planning to undergo application modernization projects. Their ability to scale up 3rd Platform adoption will require changes to IT operation that bring agility and overcome siloed legacy systems. This need for speed will bring the first big wave of DevOps adoption in the region and will make agile development the de facto norm.
Internet companies have run amok with our personal data, and people aren’t entirely sure what to do about it, judging from the results of a new survey.
More than 90 percent of Americans feel they’ve lost control over how their personal information is collected and used by companies, particularly for advertising purposes, according to the results of a survey by the Pew Research Center, published Wednesday.
Eighty percent expressed concern over how third parties like advertisers accessed the data they share on social media sites. Pew did not gather the names of which sites specifically respondents meant, but you could likely venture a guess.
The survey, which polled 607 adults online, was the Washington, D.C.-based think tank’s first in a series to tackle Americans’ views toward privacy after the leaks around government surveillance made by Edward Snowden last year.
The majority of respondents did indeed say that people should be concerned about whether the government is listening in on their phone calls, or viewing their online communications and other sensitive data.
But beyond government surveillance, the findings also reflect people’s attitudes amid the increasing sophistication by which Internet companies leverage people’s data for advertising.
“It’s a bundle of concerns,” said Lee Rainie, one of Pew’s lead researchers on the project, in an interview. “It’s partly surveillance, it’s partly tracking, and this generalized sense that I’m losing control of my identity and my data,” he said.
The constant flood of stories related to data breaches, whether it’s at Target, Snapchat, or P.F. Chang’s, don’t help either.
But voicing concern about the level of access companies, governments and other groups have to data is one thing; taking action in response is another.
Some respondents said they have taken actions to protect their privacy, like using a pseudonym, but a majority of respondents agreed that achieving anonymity online is not possible.
People’s concerns around privacy might be part of the trade-off in using a free service. Some 55 percent of respondents said they were willing to share “some information about myself with companies in order to use online services for free.”
Something very special happened at last month’s Dreamforce conference in San Francisco. Will.i.am, one of the world’s biggest pop stars, launched his new smartband wearable device, the i.am.PULS – and the worlds of music, fashion, technology, mainstream and enterprise culture well and truly collided.
“I’m an ideas guy,” he said, and it’s true that will.i.am has been extremely busy in recent years investing in game-changing technologies as well as producing award-winning music. A true innovator, he contributed to the massive success of Beats headphones and developed the concept behind Ekocycle, Coca-Cola’s sustainable living brand.
This is a man whose vision of the future, as he explained on-stage with Marc Benioff earlier this year, has been influenced heavily by the pace of innovation in technology. Echoing Facebook’s mantra that technology’s evolutionary journey is only “1% finished,” will.i.am argued that the tech landscape will be “unrecognisable” in ten years’ time: “The thing on your wrist that talks to a phone…is not the future, it’s a starting point.”
The next revolution in connected devices
Shipments of wearables are projected to reach almost 112 million units in 2018, up from less than 20 million this year (IDC). As wearables proliferate, they will add to a vast universe of interconnected, smart devices. And when the inevitable take-off of wearables does arrive, the opportunities for brands will reach a new stratosphere as they look to own the customer journey.
Wearables are set to provide marketers with the purest view of the customer yet, in terms of the volume and immediacy of the data gathered. The rise of mobile and social prompted talk of always-on marketing, and the proliferation of wearables will further enable marketers to deliver the right message to the right user at the right time. Even better, because wearables are, by nature, deeply integrated into a daily lifestyle, marketers have an opportunity to learn more about their users than ever before.
Imagine what this could mean for your brand. How might you exploit this massive opportunity to improve customer service and make marketing messages more relevant?
Data, data, data
The key to cracking wearable tech for marketing lies in – you guessed it – data. If Mark Zuckerberg’s law (the rate of increase for social sharing) is accurate, in 10 years there will be more pieces of content shared every day (95 billion) than we currently share each month (89 billion).
Of course, as marketers we’ve been talking for a few years now about the importance of data in digital marketing. The challenge comes in tracking, filtering and measuring this data so that you have a true single view of the customer. The need to effectively leverage your customer data – including social data – is only going to increase as the number of consumer devices increases, and as wearables move into mainstream adoption. This will be crucial to providing the deeper levels of personalisation that customers now expect.
We have asked the IDG Mobile Advisory Board why mobile marketing is crucial in the advertising mix. This is what Christopher Carmichael, Director of Media & Digital Marketing at Hewlett-Packard, said…
Mobility is one of HP’s core solutions within what we call the “New Style of IT” along with Cloud, Security and Big Data. Mobile is a trend that is not going to go away, and is equally important across the Consumer and Business worlds. From a marketing perspective, it’s early days still for the medium. And, as it is so often the case with a new medium or technology, people resent being interrupted with advertising at first, but gradually over time they start to accept it.
For mobile marketing, that means 3 things:
1. Some of the processes surrounding the medium are not there yet – the mechanics of planning and buying, the metrics and reporting etc.
2. Tech favours interruption rather than engagement or adding value in some way for consumers.
3. Brands need to take care not to annoy people, and to use the medium thoughtfully in a way that adds value.bit of thought can go a long way!
See what James Foulkes, Co-Founder of Kingpin Communications, says about mobile marketing…
See what Jon Hook, Head of Mobile at Mediacom International and Mediacom Beyond Advertising, says about mobility for business…