Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

Renewable energy – Is it a common man’s cup of tea?
I came across an article on a young graduate in renewable energy engineering. The fellow was doing technical sales and marketing jobs for renewable energy products though he felt that as a graduate, he ought to be doing more than just sales. His, sentiments, I can relate with but again thinking about the field of renewable energy, how many people understand what it is, its importance/ benefits, how to acquire it, its installation, costs etc.? Renewable energy is energy generated from natural resources. The renewable energy sources include sunlight, wind, rain, tides, geothermal heat and various forms of biomass. These sources are renewable naturally and continuously replenished, therefore this energy cannot be exhausted. Renewable energy technologies range from solar power, wind power, hydroelectricity/micro hydro, biomass and bio-fuels for transportation. Back to the aspiring young professional who felt that his place in the renewable energy sector lies in doing strategies and coming up with new products-the advice fronted to him was that doing technical sales is the best job for engineers, as it helps them impact on users of their products. Sales entail interacting with customers and knowing their needs so that the product features can be enhanced to suit the customer?s needs. Now, that is brilliant and accurate advice. It is however important to take into consideration that renewable energy is not a common man?s cup of tea and right now the focus all over the world is to build green economies. To me the need for more and more people to understand the benefits, savings and cost of renewable energy cannot be overemphasised. Effort should be made to keep marketing of renewable energy products/ services simple and conversational by avoiding use of acronyms or jargon explaining about operational details. More impact can be made if a marketing rather than technical sales approach is used. Technical sales have been described as boring (can be used as a sleeping aid), tends to use extensive vocabulary, jargon and acronyms that product users cannot relate with and tends to discuss the products technical aspects as opposed to the benefits to the customer. Fun should be created out of all this by making things simple and demonstrating cost savings and benefits of renewable energy.
Total Quality Management

Total Quality Management (TQM) is another business management approach that focuses on the involvement of all members of the organisation to participate in improving processes, products, services, and the culture in which they work in. It is important that every team member realises how each individual and each activity affects, and in turn is affected by, others.

With the use of combined quality and management tools, TQM also aims to reduce losses brought about by wasteful practices, a common concern in most companies. Using the TQM strategy, business would also be able to identify the cause of a defect, thereby preventing it from entering the final product.

Deming’s 14 Points

At the core of the Total Quality Management concept and implementation is Deming’s 14 points, a set of guidelines on quality as conceptualised by W Edwards Deming, one of the pioneers of quality. Deming’s 14 points are as follows:

  1. Create constancy of purpose for improving products and services.
  2. Adopt the new philosophy.
  3. Cease dependence on inspection to achieve quality.
  4. End the practice of awarding business on price alone; instead, minimise total cost by working with a single supplier.
  5. Improve constantly and forever every process for planning, production and service.
  6. Institute training on the job.
  7. Adopt and institute leadership.
  8. Drive out fear.
  9. Break down barriers between staff areas.
  10. Eliminate slogans, exhortations and targets for the workforce.
  11. Eliminate numerical quotas for the workforce and numerical goals for management.
  12. Remove barriers that rob people of pride of workmanship, and eliminate the annual rating or merit system.
  13. Institute a vigorous program of education and self-improvement for everyone.
  14. Put everybody in the company to work accomplishing the transformation.

But if you were to reduce to bare bones the TQM philosophy from Deming’s 14 points, it would all come down to two simple goals:

  1. To make things right the first time; and
  2. To work for continuous improvement.

As with all other quality management process, the end goal is to be able to offer products and services that meet and even exceed customer’s expectations.

Find out more about our Quality Assurance services in the following pages:

2015 ESOS Guidelines Chapter 7, 8 & 9 – Sign-Off, Compliance & Appeals

This is the final chapter in our series of short posts summarising the quite complex ESOS guidelines (click on ?Comply with ESOS? to see the details). This one addresses the legalities to follow to complete your report – and how to appeal if you are not happy with any of the Environment Agency?s decisions.

  1. Director Sign-Off

This is by no means an easy ride. Confirmation of the work at individual or lead assessor level locks the company into the penalty cycle in the event there are significant irregularities. By signing off the assessment, the board level director(s) # agree that they have

  • Reviewed the enterprise?s ESOS recommendations
  • Believe the enterprise is within the scope of the scheme
  • Believe the enterprise is compliant with the scheme
  • Believe the information provided is correct

Having an internal assessor requires a second board-level signature.

  1. Compliance

You report compliance on the internet. This is free and you can do it at any time within the deadline. You can dip in and out of the process as many times as you wish, but must use the link in the receipting email. While this is something a board member must do, there is no reason why the lead assessor should not complete the basics. The online compliance notification addresses the following topics:

  • The ESOS contact person in the enterprise
  • Any aggregation / dis-aggregation during the period
  • The names and contact details of the lead assessor
  • The proportion of energy consumption per compliance route

The Environment Agency will acknowledge receipt. This does not constitute acceptance. You should keep the ESOS evidence pack in a safe place with at least one backup elsewhere.

  1. Compliance & Enforcement Issues

In the event the Environment Agency decides your enterprise has not met ESOS requirements, it may either (a) issue a compliance notice with instructions, or (b) apply one of the following civil penalties:

  • A fine of up to ?5,000 for failure to maintain records
  • A fine of up to ?50,000 for failure to undertake an energy audit
  • A fine of up to ?50,000 for a false or misleading statement

Any enterprise has the right of appeal against government decisions. In the case of ESOS, this is via:

  • The First-Tier Tribunal if your enterprise is England, Wales or off-shore based
  • The Scottish Minister if your enterprise is based in Scotland
  • The Planning Commission if your enterprise is Northern Ireland-based

The notice you appeal against will supply details of the appeal steps to take.

This blog and its companion chapters concerning the ESOS Guidelines as amended 2015 are with compliments of ecoVaro. We are the people who break ESOS data into manageable chunks of information, so that board-level directors have greater confidence in what they sign.

Ready to work with Denizon?

Contact Us Today