Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

Spreadsheet Woes – Burden in SOX Compliance and Other Regulations

End User Computing (EUC) or end User Developed Application (UDA) systems like spreadsheets used to be ideal ad-hoc solutions for data processing and financial reporting. But those days are long gone.

Today, due to regulations like the:

  • Sarbanes-Oxley (SOX) Act,
  • Dodd-Frank Act,
  • IFRS (International Financial Reporting Standards),
  • E.U. Data Protection Directive,
  • Basel II,
  • NAIC Model Audit Rules,
  • FAS 157,
  • yes, there?s more ? and counting

a company can be bogged down when it tries to comply with such regulations while maintaining spreadsheet-reliant financial and information systems.

In an age where regulatory compliance have become part of the norm, companies need to enforce more stringent control measures like version control, access control, testing, reconciliation, and many others, in order to pass audits and to ensure that their spreadsheets are giving them only accurate and reliable information.

Now, the problem is, these control measures aren’t exactly tailor-made for a spreadsheet environment. While yes, it is possible to set up a spreadsheet and EUC control environment that utilises best practices, this is a potentially expensive, laborious, and time-consuming exercise, and even then, the system will still not be as foolproof or efficient as the regulations call for.

Testing and reconciliation alone can cost a significant amount of time and money to be effective:

  1. It requires multiple testers who need to test spreadsheets down to the cell level.
  2. Testers will have to deal with terribly disorganized and complicated spreadsheet systems that typically involve single cells being fed information by other cells in other sheets, which in turn may be found in other workbooks, or in another folder.
  3. Each month, an organisation may have new spreadsheets with new links, new macros, new formulas, new locations, and hence new objects to test.
  4. Spreadsheets rarely come with any kind of supporting documentation and version control, further hampering the verification process.
  5. Because Windows won’t allow you to open two Excel files with the same name simultaneously and because a succession of monthly-revised spreadsheets separated by mere folders but still bearing the same name is common in spreadsheet systems, it would be difficult to compare one spreadsheet with any of its older versions.

But testing and reconciliation are just two of the many activities that make regulatory compliance terribly tedious for a spreadsheet-reliant organisation. Therefore, the sheer intricacy of spreadsheet systems make examining and maintaining them next to impossible.

On the other hand, you can’t afford not to take these regulations seriously. Non-compliance with regulatory mandates can have dire consequences, not the least of which is the loss of investor confidence. And when investors start to doubt the management’s capability, customers will start to walk away too. Now that is a loss your competitors will only be too happy to gain.

Learn more about our server application solutions and discover a better way to comply with regulations.

More Spreadsheet Blogs


Spreadsheet Risks in Banks


Top 10 Disadvantages of Spreadsheets


Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry


How Internal Auditors can win the War against Spreadsheet Fraud


Spreadsheet Reporting – No Room in your company in an age of Business Intelligence


Still looking for a Way to Consolidate Excel Spreadsheets?


Disadvantages of Spreadsheets


Spreadsheet woes – ill equipped for an Agile Business Environment


Spreadsheet Fraud


Spreadsheet Woes – Limited features for easy adoption of a control framework


Spreadsheet woes – Burden in SOX Compliance and other Regulations


Spreadsheet Risk Issues


Server Application Solutions – Don’t let Spreadsheets hold your Business back


Why Spreadsheets can send the pillars of Solvency II crashing down

?

Advert-Book-UK

amazon.co.uk

?

Advert-Book-USA

amazon.com

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
How the Dodd-Frank Act affects Investment Banking

The regulatory reform known as the Dodd-Frank Act has been hailed as the most revolutionary, comprehensive financial policy implemented in the United States since the years of the Great Depression. Created to protect consumers and investors, the Dodd-Frank Act is made up of a set of regulations and restrictions overseen by a number of specific government departments. As a result of this continuous scrutiny, banks and financial institutions are now subject to more-stringent accountability and full-disclosure transparency in all transactions.

The Dodd-Frank Act was also created to keep checks and balances on mega-giant financial firms that were considered too big to crash or default. This was especially deemed crucial after the collapse of the powerhouse financial institution Lehman Brothers in 2008. The intended result is to bring an end to the recent rash of bailouts that have plagued the U.S. financial system.

Additionally, the Dodd-Frank Act was created to protect consumers from unethical, abusive practices in the financial services industry. In recent years, reports of many of these abuses have centered around unethical lending practices and astronomically-high interest rates from mortgage lenders and banks.

Originally created by Representative Barney Frank, Senator Chris Dodd and Senator Dick Durbin, the Dodd-Frank Wall Street Reform and Consumer Protection Act, as it is officially called, originated as a response to the problems and financial abuses that had been exposed during the nation’s economic recession, which began to worsen in 2008. The bill was signed into law and enacted by President Obama on July 21, 2010.

Although it may seem complicated, the Dodd-Frank Act can be more easily comprehended if broken down to its most essential points, especially the points that most affect investment banking. Here are some of the component acts within the Dodd-Frank Act that directly involve regulation for investment banks and lending institutions:

* Financial Stability Oversight Council (FSOC): The FSOC is a committee of nine member departments, including the Securities and Exchange Commission, the Federal Reserve and the Consumer Financial Protection Bureau. With the Treasury Secretary as chairman, the FSOC determines whether or not a bank is getting too big. If it is, the Federal Reserve can request that a bank increase its reserve requirement, which is made up of funds in reserve that aren’t being used for business or lending costs. The FSOC also has contingencies for banks in case they become insolvent in any way.

? The Volcker Rule: The Volcker Rule bans banks from investing, owning or trading any funds for their own profit. This includes sponsoring hedge funds, maintaining private equity funds, and any other sort of similar trading or investing. As an exception, banks will still be allowed to do trading under certain conditions, such as currency trading to circulate and offset their own foreign currency holdings. The primary purpose of the Volcker Rule is to prohibit banks from trading for their own financial gain, rather than trading for the benefit of their clients. The Volcker Rule also serves to prohibit banks from putting their own capital in high-risk investments, particularly since the government is guaranteeing all of their deposits. For the next two years, the government has given banks a grace period to restructure their own funding system so as to comply with this rule.

? Commodity Futures Trading Commission (CFTC): The CFTC regulates derivative trades and requires them to be made in public. Derivative trades, such as credit default swaps, are regularly transacted among financial institutions, but the new regulation insures that all such trades must now be done under full disclosure.

? Consumer Financial Protection Bureau (CFPB): The CFPB was created to protect customers and consumers from unscrupulous, unethical business practices by banks and other financial institutions. One way the CFPB works is by providing a toll-free hotline for consumers with questions about mortgage loans and other credit and lending issues. The 24- hour hotline also allows consumers to report any problems they have with specific financial services and institutions.

? Whistle-Blowing Provision: As part of its plan to eradicate corrupt insider trading practices, the Dodd-Frank Act has a proviso allowing anyone with information about these types of violations to come forward. Consumers can report these irregularities directly to the government, and may be eligible to receive a financial reward for doing so.

Critics of the Dodd-Frank Act feel that these regulations are too harsh, and speculate that the enactment of these restrictions will only serve to send more business to European investment banks. Nevertheless, there is general agreement that the Dodd-Frank Act became necessary because of the unscrupulous behaviour of the financial institutions themselves. Although these irregular and ultimately unethical practices resulted in the downfall of some institutions, others survived or were bailed out at the government’s expense.

Because of these factors, there was more than the usual bi-partisan support for the Dodd-Frank Act. As a means of checks and balances, the hope is that the new regulations will make the world of investment banking a safer place for the consumer.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
Enhance and Streamline IT Processes

You can’t be assured of a competitive advantage by just buying the latest technology. Your top competitor can easily match that feat by simply spending as much on the same tools. To be always at least a step ahead, you’ll need to perform tweaks on your IT processes aligned with the strengths of your organisation.

IT solutions are like a pair of sneakers. If they fit perfectly, they’ll help you run the extra mile. If they don’t, you can develop blisters faster than you can reach a single mile.

In all our efforts to enhance and streamline your IT processes, we’ll start by looking at all your logistical advantages, limitations, and objectives to determine which technologies suit you best. Once we’ve obtained them, we’ll perform the appropriate customisation to make them perform optimally under the conditions unique to your organisation.

Below are just some of the enhancements we can apply to your organisation:

  • Put up application and systems monitoring to identify bottlenecks and underutilised resources in your IT infrastructure.
  • Propose areas where you can plough back the generated savings to further improve your ROI.
  • Take scalability into consideration when pushing for certain IT investments to ensure that the IT solution will work for your organisation not only today but even as your organisation grows.
  • Introduce mobile-capable enterprise-class IT solutions that allow seamless collaboration between team members working at different locations on the globe so that pressing matters can be resolved and decisions can be arrived at as quickly as possible.
  • Integrate Business Intelligence into your IT system so that massive collections of data can be processed into insightful information which managers can draw on to make intuitive decisions.
  • Introduce avant-garde solutions, like virtualisation and infrastructure sharing, which may require large scale changes but can also significantly reduce operational costs.

Find out how we can increase your efficiency even more:

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?