Big Data for Everyone

“humanity [is] now creating as much information every two days as it did from the dawn of civilization up to 2003—every 48 hours, 5 exabytes, or 5 quintillion bytes (5 followed by 18 zeros).”

Big Data: What is it?

It’s a commonly used term but one with many definitions. In its simplest iteration, big data is a concise term for the aggregation of smaller data – things like the information from a web form, website activity tracking, and larger things like surveillance feeds and video production. All of these individual data streams coagulate into a huge, I mean really huge, set of information known as “big data.” To give a scale of this, Eric Schmidt, former Google CEO, told attendees at the Techonomy conference in 2010 that “humanity was now creating as much information every two days as it did from the dawn of civilization up to 2003—every 48 hours, 5 exabytes, or 5 quintillion bytes (5 followed by 18 zeros).”

That is a massive amount of data. And its impact comes down to one thing: access. Access gives information, information begets decision-making, good decision-making begets competitiveness, competitiveness begets better products, and better products beget superiority and recurring revenues.

The access to data has changed and will continue to change the way the connected society conducts businesses – from making life-and-death decisions on what defense missile to build down to the trifling estimation of your buying sandalwood over lavender scented shaving cream.

Access to this range of information creates major value when analyzed. McKinsey & Company has identified five ways big data does this by :

  1. Making information transparent, thus spurring productivity by many users
  2. Finding performance metrics for “everything from product inventories to sick days” which fuels decision making and forecasting
  3. Identifying customer segmentations that drive product customization
  4. Deploying analytics for decision making
  5. Improving future product and service deployments by tracking sensors and inputs

These value points are broad enough to fit over any category of high-security products and services, from owners, to architects, to general contractors, to consultants, to integrators, to vendors. Each level of business has potential to utilize big data to improve its position and competitiveness.

How data improves our products and services

The ability to analyze streams of data has altered the way many of us consume information and products. It’s likely our exposure to ideas and products is a result of carefully planned algorithms that track our relevant and useless conversations, contributions, searches, and inputs. The results from tracking tell marketers where to suggest their products and who’s looking for them.

This same concept is used by SECs in their software development. Tracking clicks gives an indication of how users employ the software and how it can be altered and enhanced. Especially useful, is user interaction during emergencies. As device types expand to tablets and even phones in some applications, these trackers are imperative evidence in destructing a crisis from multiple angles. An SEC will design their software according to specification and owner requirements but if a certain behavior is observed in the software during times of crisis, click tracking will directly identify a preferred operation.

This opinion-free tracking is also cogent protection for software providers who are questioned after an emergency for faulty equipment. Nearly 100% of the time, emergencies result because of human error and tracking proves to be the garrison for firms who find themselves in questioning.

More commonly, though, in high-security environments, big data is applied to the data accumulated through video captured in facilities and its storage. Storage and clarity requirements continuously increase, creating a demand for robust data management systems, sometimes to the point of having an on-site data center.

Storage and TCO

This massive amount of data has to go somewhere. And not just go, it has to live somewhere. It’s a big deal for this industry and it’s a big deal for agencies like the NSA who is building a $2 billion dollar facility in Utah whose footprint spans a stately 6.5 million square feet. That’s a lot of storage space considering multiple terabytes can fit on a solid state drive the size of a credit card. The NSA, like the every industry must sort through two types of data: structured and unstructured. Structured data are the inputs into things like web forms to populate databases and as such, are highly organized information. Unstructured data, on the other hand are everything else – photos, graphics, videos, streaming information, webpages, pdfs, email, Word documents, PowerPoints, etc, which is the majority of data we create. Unstructured data takes significantly more space to store than structured data.

Though our industry doesn’t require nearly the storage as the NSA, our demands on saving and retrieving unstructured information are great. They’re large enough to attract big data storage companies to the market and the good ones are bringing with them major efficiencies that reduce TCO, or Total Cost of Ownership.

Engrained in TCO are the costs for product acquisition, maintenance, support, security, training, and other fully burdened costs. The goal, of course, is to have the lowest cost of ownership for the most reliable system architecture. It’s imperative to have a data expert on a project planning team who will direct planning to a scalable, multi-dimensional infrastructure with the full consideration of future costs and maintenance. Many think an acquisition like storage is a once every five years purchase when best practices actually prove it to be a more active asset. As a matter of efficiency, a sophisticated storage provider’s preemptive technology saves owners from unexpected expenditures in the long run.

As an industry whose projects and daily operations are funded based on metrics, big data is inescapable and is quickly securing its space in the market. The pent-up demand for creating and storing information is contributing to the massive amounts of data we create daily, and the access to it will improve our businesses, operations, and dexterity.