Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

Project Management

In a cutthroat market, where the competition is constantly on the attack to break into your market share, implementing a project-based system can give your organisation the necessary tools to be more efficient and agile.

However, rapidly changing consumer demands, technologies and other factors make it ever more difficult to generate a strategic advantage from projects, let alone develop one. Also since a large organisation can easily end up having to manage multiple projects at the same time, the new management paradigm can appear too complex.

What your company really needs is the expertise that can guide you starting from conception and planning, down through procurement and execution in order to maximise whatever resources you have. Each move must be well thought out so that there are clear goals and objectives as well as methods to achieve them.

Programme Management

Are you running multiple projects pointing to an overall strategic direction? Then you’ll need more than just a “scaled-up” version of project management to make sure every component’s work effort is well coordinated to achieve your enterprise’s desired outcomes.

Through our expertise in programme management, we’ll work with your stakeholders, executives and clients to achieve the following:

  • Design a well-articulated management structure and clearly define decision-making roles & responsibilities – This will ensure decisions are made rapidly with zero to minimal overlapping issues and to promote a unified, well-synchronised advance towards the common objective.
  • Set objectives then make sure they are met by guiding your key personnel in coordinating activities across projects.
  • Design or utilise existing financial models such that they adhere to your enterprise’s financial policies.
  • Develop procedures for reporting expenditures specific to the programme.
  • Establish the programme infrastructure, including
    • The appropriate technical environment and tools (e.g. hardware, software, communication, and other IT-related items)
    • IT staff and administrators
  • Evaluate your enterprise’s current IT architecture to determine whether it will suffice to achieve your objectives. If it doesn’t, propose options you can take to meet what is required.
  • Plan out activities that should take place in different levels in the organisation.
  • Implement a periodic review of the programme progress as well as of interim results to ensure everything is aligned with the strategic outcome.

Programme and Project Reviews

Whether we’ve helped you set up your programme or you did it on your own, time will come when you’ll need to know whether everything is going as planned. If it appears like the entire programme is going smoothly, chances are, something’s going awfully wrong somewhere. Remember, even the most well-planned projects and programmes are still under the mercy of unforeseen variables.

We’ve got highly specialised reviews for either projects or an entire programme. We’ll be able to provide you answers to questions like:

  • Are all projects aligned with the programme’s intended direction?
  • Are the people working on your projects as focused with the business rationale as they have been with meeting deadlines and utilising resources?
  • Where are your risks and exposures? How can they be remedied?
  • Is the project viable at all?

We understand how your staff would want to function normally as quickly as possible. Rest assured, our programme and project reviews are conducted swiftly and efficiently so that both interruptions and oversights are brought to a minimum.

After we’re done, you can expect a detailed quantitative assessment of your programme and/or projects’ status.

Basically, we’re not here to find mistakes; we’re here to help you find ways to correct them. If a project rescue is required, we’ll be the first to lend a hand.

Project Rescue

Believe it or not, many of our clients approached us not before or during their project’s planning stages. But rather, after having gone through sloppy execution, when they end up losing control. In other words, we’re usually at the receiving end of the distress signal, after they’ve punched the panic button.

While obviously this isn’t the ideal time to seek the aid of any expert because it means you’ve incurred unnecessary losses already, all is not yet lost. If the appropriate remedial actions are taken in a timely manner, you can still achieve highly acceptable end results.

In fact, in most of our experiences with project rescue operations, we’ve been able to put projects back on track – just the way the planners wanted them to be. We’ll also help you devise airtight strategies to prevent your project from going astray again.

At the end of our project rescue,

  • You’ll regain complete control
  • Milestones will be reached as planned
  • Requirements will be accomplished, and
  • The project will be realigned with ideal business directions

Project Governance Processes

Constructing a firm underlying structure is essential in any organisation. So before we’ll institute project management, we’ll do the following first.

  • Set up a PMO or Project Management Office to ensure, among others, that
    • Utilisation of facilities, budgets, technical support and other resources will be well coordinated
    • Work products can be tracked and reviewed
    • Issues regarding methodology and processes will be given appropriate attention
    • Training can be organised
    • Project management discipline be instilled in the IT department
  • Establish a steering committee to oversee the implementation of IT and business strategies
  • Fill up slots for a project manager, IT executive and a business sponsor and define the roles of each
  • Infuse project management practices to all affected units of the enterprise

Establishing PMOs, steering committees and other management structures is the easy part. Many organisations spend so much in order to create the structures related to project management, only to find out later that the effort has been all for naught. That’s why we won’t end there. Our objectives will therefore include the following:

  • To plant and cultivate an environment appreciative of project governance i.e. one that does not project it as just a bunch of bureaucratic processes and protocols.
  • To establish an organisational culture that starts at the top.
  • To make everyone involved understand that the power of project governance still lies in the hands of those who will ultimately implement it.

A project-driven enterprise is never propelled by a single project. Since multiple projects require a more complex governing structure, you’ll need to understand the intricacies of programme management.

Web Design and Development

The first few seconds of a first-time website visitor is very crucial. If they don’t like what they see or if they think it takes too long just to load what they’re supposed to see, chances are, that would be the last time you’d ever catch them on your site.

Therefore, striking a balance between your website’s appearance and its loading speed is important for first impressions. Once you’ve captured the visitor’s attention, the next objective is to keep them glued long enough for them to browse through your merchandise. It is at this point that the benefits of a well organised and highly intuitive graphical user interface come into play.

An excellent combination of stylish web design and sharp web development can play a major role in lowering bounce rates and increasing returning visitors. We see to it that our web designers and developers not only excel at what they do individually, but also understand the interplay between their individual creations and how it affects the overall appeal of the website.

This is what you can expect from our brand of web design and development:

  • Conversion-motivated web design. Since we understand that your primary motivation for entering into the eCommerce arena is to turn torrential web traffic into sales, we’ve put conversion as a primary consideration in our web designs.
  • SEO-friendly content. First-time visitors don’t reach your site because they entered your URL somewhere. Rather, they must have stumbled upon your links on search engine results or on other websites.
  • Engaging web content. Because excellent graphics alone can’t sell products but engaging web content can, we invest in excellent copywriters.
  • Visitor-friendly user interface. Before a visitor will ever read content on the current and succeeding pages, they’ll need to interact with your site’s UI first. We’ll make sure your user interface is visually appealing enough to invite visitors to click on your buttons.
  • Superior expertise in web development technologies. Our web developers are certified experts in web related technologies including Javascript, AJAX, SQL, PHP, CSS, Java, Silverlight, CMSes, and Magento, among others. Thus, we can offer extreme flexibility and scalability in our web development services.

See more related services

Spreadsheet Risks in Banks

No other industry perhaps handles such large volumes of critical financial data more than the banking industry. For decades now, spreadsheets have become permanent fixtures in the front-line reporting tool sets of banks, providing organised information when and where needed.

But as banks enter into a period of heightened credit risks, elevated levels of fraud, and greater regulatory scrutiny, many are wondering if continued reliance on spreadsheets is a wise decision for banks today.

The downfall of Lehman Brothers which eventually led to its filing for Chapter 11 bankruptcy protection on September 15, 2008, served as a wake up call for many institutions across the globe to make a serious examination of their own risk management practices. But would these reforms include evaluating the security of user developed applications (UDAs), the most common of which are spreadsheets, and putting specific guidelines as to when they can – or cannot be – used?

Banks and Spreadsheet Use

Banks have been known to utilise spreadsheets systems for many critical functions because most personnel are well-acquainted with them, and the freedom of being able to develop customised reports without needing to consult with the IT department offers flexibility and convenience. In fact, more than having a way to do financial budgeting and analysing customer profitability, even loan officers and trade managers have become reliant on spreadsheets for risk management reporting and for making underwriting decisions.

But there are more than a few drawbacks to using spreadsheets for these tasks, and the sooner bank executives realise these, the sooner they can adopt better solutions.

General Limitations

Spreadsheets are far from being data base systems and yet more often than not, they are expected to act as such, with figures constantly added and formulas edited to produce the presumably right set of reports.

In addition, data integrity is always a cause for concern as most values in spreadsheets are entered as manual inputs. Even the mere misplacement of a comma or a negative sign, or an inadvertent ?edit? to a formula can also be a source of significant changes in the outcome.

Confidentiality risk is also another drawback of the use of spreadsheets in banks as these tools do not have adequate?access controls to limit access to only authorised individuals. Pertinent financial information that fall into the wrong hands can lead to a whole new set of problems including the possibility of fraud.

Risks in Trading

For trading transactions, spreadsheets can prove to be of immense use – but only for small market volumes. As trade volumes increase and the types vary, spreadsheets are no longer a viable solution and may likely become more of a hindrance, with calculations taking longer in the face of bigger transaction amounts and growing transaction data.

And in trading, there is always the need for rigorous computational functions. Computing for the Value at Risk (VaR) for large portfolios for instance, is simply way beyond the capabilities of spreadsheets. Banks that persist in using them are increasing the risk of loss on those portfolios. Or, they can be opening up?opportunities for fraud?as Allied Irish Bank (in the case of John Rusnak – $690 million) learned the hard way.

Risks in Underwriting

Bankers who use spreadsheets as their main source of information for underwriting procedures also face certain limitations. Loan transactions require that borrowers? financial data be centralised and easily accessible to risk officers and lending officers involved in making decisions. With spreadsheets, there is no simple and secure way of doing that. Information can be pulled from different sources – individual tax returns, corporate tax documents, partnership documents, audited financial statements – hence there is difficulty in verifying that these reports adhere to underwriting policies.

Spreadsheet control and monitoring

Financial institutions which are having difficulty weaning themselves from the convenience and simplicity that spreadsheets offer are looking for possible control solutions. Essentially, they want to find ways that allow them to continue using these UDAs and yet somehow eliminate the?spreadsheet risks?and limitations involved.

Still, the debate goes back and forth on whether adequate control measures can be implemented on spreadsheets so that that the risks are mitigated. Many services have come forward to herald innovative solutions for better spreadsheet management. But at the end of the day, there really is no guarantee that such solutions would suffice.

More Spreadsheet Blogs


Spreadsheet Risks in Banks


Top 10 Disadvantages of Spreadsheets


Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry


How Internal Auditors can win the War against Spreadsheet Fraud


Spreadsheet Reporting – No Room in your company in an age of Business Intelligence


Still looking for a Way to Consolidate Excel Spreadsheets?


Disadvantages of Spreadsheets


Spreadsheet woes – ill equipped for an Agile Business Environment


Spreadsheet Fraud


Spreadsheet Woes – Limited features for easy adoption of a control framework


Spreadsheet woes – Burden in SOX Compliance and other Regulations


Spreadsheet Risk Issues


Server Application Solutions – Don’t let Spreadsheets hold your Business back


Why Spreadsheets can send the pillars of Solvency II crashing down

?

Advert-Book-UK

amazon.co.uk

?

Advert-Book-USA

amazon.com

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?