Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

Free to download, use and contribute to

Everything you need ?in the box? to get started

Distributed across multiple fire-walled computers

Fast processing of data held in efficient cluster nodes

Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

Spreadsheet Woes – Burden in SOX Compliance and Other Regulations

End User Computing (EUC) or end User Developed Application (UDA) systems like spreadsheets used to be ideal ad-hoc solutions for data processing and financial reporting. But those days are long gone.

Today, due to regulations like the:

Sarbanes-Oxley (SOX) Act,
Dodd-Frank Act,
IFRS (International Financial Reporting Standards),
E.U. Data Protection Directive,
Basel II,
NAIC Model Audit Rules,
FAS 157,
yes, there?s more ? and counting

a company can be bogged down when it tries to comply with such regulations while maintaining spreadsheet-reliant financial and information systems.

In an age where regulatory compliance have become part of the norm, companies need to enforce more stringent control measures like version control, access control, testing, reconciliation, and many others, in order to pass audits and to ensure that their spreadsheets are giving them only accurate and reliable information.

Now, the problem is, these control measures aren’t exactly tailor-made for a spreadsheet environment. While yes, it is possible to set up a spreadsheet and EUC control environment that utilises best practices, this is a potentially expensive, laborious, and time-consuming exercise, and even then, the system will still not be as foolproof or efficient as the regulations call for.

Testing and reconciliation alone can cost a significant amount of time and money to be effective:

It requires multiple testers who need to test spreadsheets down to the cell level.
Testers will have to deal with terribly disorganized and complicated spreadsheet systems that typically involve single cells being fed information by other cells in other sheets, which in turn may be found in other workbooks, or in another folder.
Each month, an organisation may have new spreadsheets with new links, new macros, new formulas, new locations, and hence new objects to test.
Spreadsheets rarely come with any kind of supporting documentation and version control, further hampering the verification process.
Because Windows won’t allow you to open two Excel files with the same name simultaneously and because a succession of monthly-revised spreadsheets separated by mere folders but still bearing the same name is common in spreadsheet systems, it would be difficult to compare one spreadsheet with any of its older versions.

But testing and reconciliation are just two of the many activities that make regulatory compliance terribly tedious for a spreadsheet-reliant organisation. Therefore, the sheer intricacy of spreadsheet systems make examining and maintaining them next to impossible.

On the other hand, you can’t afford not to take these regulations seriously. Non-compliance with regulatory mandates can have dire consequences, not the least of which is the loss of investor confidence. And when investors start to doubt the management’s capability, customers will start to walk away too. Now that is a loss your competitors will only be too happy to gain.

Learn more about our server application solutions and discover a better way to comply with regulations.

More Spreadsheet Blogs

Spreadsheet Risks in Banks

Top 10 Disadvantages of Spreadsheets

Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry

How Internal Auditors can win the War against Spreadsheet Fraud

Spreadsheet Reporting – No Room in your company in an age of Business Intelligence

Still looking for a Way to Consolidate Excel Spreadsheets?

Disadvantages of Spreadsheets

Spreadsheet woes – ill equipped for an Agile Business Environment

Spreadsheet Fraud

Spreadsheet Woes – Limited features for easy adoption of a control framework

Spreadsheet woes – Burden in SOX Compliance and other Regulations

Spreadsheet Risk Issues

Server Application Solutions – Don’t let Spreadsheets hold your Business back

Why Spreadsheets can send the pillars of Solvency II crashing down

amazon.co.uk

amazon.com

Contact Us

(+353)(0)1-443-3807 – IRL
(+44)(0)20-7193-9751 – UK

Firewalls

There are two main reasons why some companies are hesitant to plug into the Internet.

They know they’ll be exposing their company data to outside attacks from malicious individuals and malware.
They fear their employees might get too many distractions: games, porn, chats, videos, and even social networking sites.

One vital component for your overall security strategy against such concerns? A firewall.

A firewall can block unauthorised access to certain Internet services from inside your organisation as well as prevent unauthenticated access from the outside. It is also used to monitor users’ activities while they were online.

In an enterprise setting, one may expect a collection of firewalls either for providing layered protection or segmenting off different units in the organisation. Some areas only need a standard line of defence while others require more restrictions. As such, certain firewalls may have different configurations compared to others.

Naturally, the more intricate an organisation’s defence requirements get, the more complex the task of monitoring, testing and configuring the firewalls becomes. That’s why we’re here to help.

We’ll evaluate your network as well as the security requirements of each department under your organisation to determine which firewall architecture is most suitable.
To achieve maximum efficiency, we’ll point out where each firewall should be positioned.
We’ll work with your key personnel to make sure all firewall configurations are set and optimised with your business rules in mind.
If a large number of firewalls are required, we’ll help you set up a firewall configuration management system.
Firewalls should be regularly tested and assessed to ensure they are in line with the organisation’s security policies. We’ll perform these routine tasks as well.

Firewalls aren’t very good at defending against sophisticated viruses. There are much better solutions for malware-related vulnerabilities, and we can help you in that regard too.

Other defences we’re capable of putting up include:

2015 – What’s ahead for UK Business?

According to reports just in, the global environment industry is down. Less money is available for what some CEO?s still see as grudge expenditure, and many U.S. agencies are seeking soft budget cuts. The UK is proving to be an exception following the announcement of ESOS, and EcoVaro does not expect the May elections will have much impact in this regard.

ESOS calls for mandatory energy assessments in companies above a certain size, and requires specific proposals to cut consumption. There is no indication of compulsory follow-through, although it is clear the Environment Agency hopes rising electricity prices and the prospect of monetary savings will do the trick.

It is an open question whether the Tory government would have interfered with commerce to this extent, were it not for the European directive that enforced it. The overall goal is to cut EU energy consumption across the board by 20% by 2020. Energy consultants are rubbing their hands in glee. EcoVaro?s response is to provide cloud-based software.

We will be interested to see how many UK companies make the first deadline of 5 December 2015, in the light of reports that half the 9,000 firms affected appear not to even know that ESOS exists. Some will no doubt pay last-minute lip service. Those with an eye on their own sustainability will grasp the Energy Saving Opportunity Scheme with both hands.

The initial ESOS deadline was always going to be a challenge. Some big corporates have stolen a march albeit egged on by green stakeholders. The next challenge comes in June 2015 with the implementation of the European Union?s ?Waste Catalogue? of hazardous substances, and rules for their disposal. We hope a new ISO 14001 will arrive soon and pull the loose threads together.

The introduction of carbon trading late this year brings further opportunities to increase profits through wise stewardship. Auditable metrics are essential for this.

EcoVaro can assist by processing your raw data. We provide this service on a virtual cloud. In return, you can get advice on optimising the quality of your graphs for presentations.