Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

Free to download, use and contribute to

Everything you need ?in the box? to get started

Distributed across multiple fire-walled computers

Fast processing of data held in efficient cluster nodes

Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

The Future of Cloud Backup and Recovery

We came across a post on Docurated that pulled together thirty-seven suggestions for the top cloud storage mistakes user companies make. Given that cloud storage seems to be the best backup solution for now at least, we decided to turn these ideas around to sense the direction cloud backup and recovery needs to take, if it is still to be relevant in say ten years? time.

Has Cloud Storage Largely Saturated the West?
It probably has. Outside of major corporates who make their own arrangements ? and SME?s that use free services by email providers ? the middle band of companies in Europe and America have found their service providers, although they may have never tested the recovery process, to see if it works.

The new gold rush in the cloud backup and recovery business is, or should be emerging markets in Asia, Africa, South America, and the Middle East. There, connectivity is brittler than over here. To be relevant in these fragile, more populous areas our cloud backup and recovery industry need to be more agile and nimble.

? It must provide a simpler service emerging commerce can afford, refresh its user interfaces in third world languages, have more accessible help, and be patient to explain how cloud storage works to newbies. In other words, it must source its call centre operators in the areas it serves.

? It must adapt to local connectivity standards, and stop expecting someone with ADSL broadband to keep up with cloud server networks running at up to 1GBPS compared to their 10MBPS at best. For user sourcing and retention purposes, these new cloud backup and recovery services must be the ones who adapt.

? It must facilitate disaster recovery simulations among its clients in calmer moments when things are going well. Are they backing up the right files, are they updating these, and are their brittle ADSL networks able to cope with their cloud service providers? upload and download speeds?

? It must develop lean and agile systems slim enough to accommodate a micro client starting out, but sufficiently elastic to transfer them seamlessly to big data performance. The Asian, African, South American, and Middle Eastern regions are volume driven, and individual economies of scale are still rare.

? It must not expect its users to know automatically what they need, and be honest to admit that Western solutions may be wrong-sized. Conversion funnels in the new gold rush are bound to be longer. Engagements there depend on trust, not elevator sales letters. Our competition in these countries already works this way.

? It must be honest and admit cloud storage is only part of the solution. To recruit and retain users it must step back to 1983, when Compuserve offered its customers 128k of disc space, and spent an amount of effort explaining how to filter what to put there.

Cloud Storage of Data is Only One Part of the Solution
Governance reports and stock certificates burn just as easily as do servers in a fire. We must not transfer bad habits to exciting new markets. We close this article with the thoughts of John Howie, COO of Cloud Security Alliance, as reported in the Docurated post we mentioned, and these apply across the globe, we believe.
There is no single most important thing to carry forward into the future of cloud backup and recovery. We must be mindful when moving data that this can be fragile too. We must also create layers of backup the way insurance companies re-insure, that make any one cloud backup and recovery business redundant if it happens.
We hold the trust of our customers in our hands but trust is delicate too. We must cease trying to make a pile of money quickly, and become more interested in ensuring that data transferred back and forth is synchronised. The cloud backup and recovery industry needs only one notorious mistake, to become redundant itself in the ten years we mentioned.

Contact Us

(+353)(0)1-443-3807 – IRL
(+44)(0)20-7193-9751 – UK

How Internal Auditors can win The War against Spreadsheet Fraud

To prevent another round of million dollar scandals due to fraudulent manipulations on spreadsheets, regulatory bodies have launched major offensives against these well-loved User Developed Applications (UDAs). Naturally, internal auditors are front and center in carrying out these offensives.

While regulations like the Sarbanes-Oxley Act, Dodd-Frank Act, and Solvency II can only be effective if end users are able to carry out the activities and practices required of them, auditors need to ascertain that they have. Sad to say, when it comes to spreadsheets, that is easier said than done.

Because spreadsheets are loosely distributed by nature, internal auditors always find it hard to: locate them, identify ownership, and trace their relationships with other spreadsheets. Now, we’re still talking about naturally occurring spreadsheets. How much more with files that have been deliberately tampered?

Spreadsheets can be altered in a variety of ways, especially if the purpose is to conceal fraudulent activities. Fraudsters can, for instance:

hide columns or rows,
perform conditional formatting, which changes the appearance of cells depending on certain values
replace cell entries with false values either through direct input or by linking to other spreadsheet sources
apply small, incremental changes in multiple cells or even spreadsheets to avoid detection
design macros and user defined functions to carry out fraudulent manipulations automatically

Recognising the seemingly insurmountable task ahead, the Institute of Internal Auditors released a guide designed specifically for the task of auditing user-developed applications, which of course includes spreadsheets.

But is this really the weapon internal auditors should be wielding in their quest to bring down spreadsheet fraud? Our answer is no. In fact, we believe no such weapon has to be wielded at all?because the only way to get rid of spreadsheet fraud is to eliminate spreadsheets once and for all.

Imagine how easy it would be for internal auditors to conduct their audits if data were kept in a centralised server instead of being scattered throughout the organisation in end-user hard drives.

And that’s not all. Because a server-based solution can be configured to have its own built-in controls, all your data will be under lock and key; unlike spreadsheet-based systems wherein storing a spreadsheet file inside a password-protected workstation does not guarantee equal security for all the other spreadsheets scattered throughout your company.

Learn more about Denizon’s server application solutions and discover a more efficient way for your internal auditors to carry out their jobs.

More Spreadsheet Blogs

Spreadsheet Risks in Banks

Top 10 Disadvantages of Spreadsheets

Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry

How Internal Auditors can win the War against Spreadsheet Fraud

Spreadsheet Reporting – No Room in your company in an age of Business Intelligence

Still looking for a Way to Consolidate Excel Spreadsheets?

Disadvantages of Spreadsheets

Spreadsheet woes – ill equipped for an Agile Business Environment

Spreadsheet Fraud

Spreadsheet Woes – Limited features for easy adoption of a control framework

Spreadsheet woes – Burden in SOX Compliance and other Regulations

Spreadsheet Risk Issues

Server Application Solutions – Don’t let Spreadsheets hold your Business back

Why Spreadsheets can send the pillars of Solvency II crashing down

amazon.co.uk

amazon.com

EU Energy Efficiency Directive & UK?s ESOS

In 2012 the European Union passed its EU Energy Efficiency Directive (EED) into law. This aims to reduce overall energy consumption by 20% by 2020. It placed an obligation on member states to pass back-to-back local legislation by June 2014.

EED Guidelines

The EED provides specific guidelines it expects member nations to address. The list is long and here are a few excerpts from it:

Large companies must use energy audits to identify ways to cut their energy consumption

Small and medium companies must be incentivised to voluntarily take similar steps

Public sector bodies must purchase energy-efficient buildings, products and services

Private energy-consumers must be empowered with information to help manage demand

Energy distributors / resellers must cut their own consumption by 1.5% annually

Legislators are free to substitute green building technology e.g. through better insulation

Every year, European governments must audit 3% of the buildings they own

Definition of Energy Audit

An energy-consumption audit is a question of measuring demand throughout a supply grid, with particular attention to individual modules and high demand equipment. While this could be an exercise repeated every four years to satisfy ESOS, it makes more sense to incorporate it into the monthly energy billing cycle.

Because energy use is not consistent but varies according to production cycle, this can produce reams of printouts designed to frustrate busy managers. ecoVaro offers an inexpensive, cloud-based analytic service that effortlessly accepts client data and returns it in the form of high-level graphic summaries.

Potential ESOS Beneficiaries

As many as 9,000 UK companies are obligated to do energy audits because they employ more than 250 employees, have a balance sheet total over ?36.5m or an annual turnover in excess of ?42m. Any smaller enterprise that finds energy a significant input cost, should also consider enlisting Ecovaro to help it to:

Obtain a better understanding of the energy side of their business

Achieve energy savings and share in a estimated ?3bn bonanza to 2030

Reduce carbon emissions to help meet their CRC commitments

More About ecoVaro

We offer web-based energy management software that helps you measure and manage energy costs. This strips data from your meters and generates personalised reports on a dashboard you control. This information helps you accurately zoom in on worthwhile opportunities. With Ecovaro on your side, ESOS truly becomes an Energy Saving OPPORTUNITY Scheme.