Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

How an EMS Can Cut Your Carbon Emissions

Your business carbon footprint is directly tied to the efficiency of its energy consumption. From the equipment used in industries, lighting and air conditioning in offices, shopping malls and other commercial buildings, the load used by everyday machines like the coffee makers in the employee breakroom, to hot water boilers in apartment complexes, how much do your processes affect the environment? Standards like the ISO 14001:2015 are being implemented to enable businesses to reduce their impact on the environment, from optimising their energy usage, minimising waste, turning to renewable power sources, all through to preventing pollution and complying with their specific regulatory requirements. How do you handle the volume of data that needs to be obtained and assessed?

Energy management systems come in to enable you to analyse your consumption, identify factors affecting your total energy use – from temperature and humidity conditions, to equipment that is causing spikes, and observe your usage patterns. That way, you can put in measures to minimise wastage while increasing your operational efficiency, reduce your carbon emissions and track your progress all the way. Here, we’ll break down how this is achieved. 

Going Green With An Energy Management System

This is a holistic approach aimed at minimising wastage and optimising energy usage. It includes:

Auditing your energy consumption

The first step is really quantifying how much energy you use, which systems are causing unnecessary load, all through to where there are inefficiencies in the facility. Which equipment has the largest impact on your bill? An energy management system allows you to view it all from one dashboard, such as with the ecoVaro EMS that takes you down to the sub-meter level.

Here, you get real-time data that is collected by the ecoVaro loggers – from electricity use, gas, water, temperature, solar power, humidity, air pressure – the readings can all be monitored. This is done 24/7, and the consumption feeds are recorded. Moreover, ecoVaro pulse data is collected every 15 minutes – which is particularly important when it comes to analysing trends over a time period, be it daily, weekly or monthly. 

Data is only useful if it can be properly analysed, right? So instead of just bombarding you with spreadsheets of numbers, the EMS displays the records into graphs and charts that are easy to comprehend – all from the same interactive interface. So, whether you’re the energy manager in the facility, or you want reports that can be shared with the CFO, owners of the business, or even staff themselves to enable them to understand the energy saving policies that you will put in place – you will be able to carry this out. 

ecoVaro gives you different ways to analyse the data from the readings that have been recommended. For instance, the heat mapping from the interface allows you to see the building’s energy use during different periods at a glance. The site-by-site analysis in particular enables the building or energy manager to assess each individual premises, from checking which block in the school is causing the energy bills to surge, the facility whose performance is falling behind, all through to the office building with the highest carbon footprint. In fact, the carbon and sustainability reports from ecoVaro EMS enables you to see the impact that your operations have. You even get to compare tariffs from the different energy suppliers, that way you can go with the option that is most suited to your situation.

Setting a baseline for your operations

This is essentially a “before/after checkpoint” that you will use to compare the effectiveness of subsequent measures that you will undertake. After making modifications to the systems in your business, you will want a clear picture of whether the new measures are actually benefiting your operations and optimising your energy efficiency, or whether they are deteriorating the performance further. The energy baseline will be critical in analysing your progress. 

Reports like the CUSUM (cumulative sum) charts on ecoVaro show you the energy performance, be it of a boiler in a factory, office building, or chain of hotels – over a set period of time. You can then compare this to the baseline, which will show you if the changes you will implement will make you savings. The heatmaps also come in handy here, showing you the energy consumption at each meter, whether it is low, medium or high compared to the baseline that has been set. The heatmaps give a quick visual to analyse resource usage.  

Creating energy targets

After understanding your energy consumption and seeing how it impacts your business, next is mapping out short- and long-term goals that you want to attain to optimise your usage and reduce your carbon footprint. 

For instance, short-term targets can include the likes of decreasing the night-time lighting load, and adjusting HVAC uptime depending on the level of activity in your business premises for the different hours of the day. 

For the long-term targets, these include setting a specific percentage average kWh reduction for the different industrial sites or buildings under your management; lowering the demand kW throughout the building by a specific range year-on-year; as well as the percentage with which you want the carbon emissions decreased annually. 

Cost efficiency also factors in. For instance, entering your current tariffs into the conversion factoring dashboard on ecoVaro will show you how your consumption translates to the bills that you receive – and even shows you what you stand to save by negotiating for new energy contracts with your utility firm.

Identifying initiatives and implementing energy saving programs

These are geared towards improving your energy efficiency and reducing your carbon footprint. They vary from one industry to the next. For instance, these can include:

Getting motion/occupancy detectors and automatic dimmers installed in the facility

These are lighting controls that enable you to save money and energy by automatically turning the lights off when they are not required (people have left the room), and reducing the light levels for those cases where full-on brightness is not needed. For instance, the dimmer controls enable variable indoor lighting, reducing the wattage and output when dimming the lightbulbs, saving energy in the process. These can be manual, or operated with sensors or timers. 

Motion sensors on the other hand will automatically turn on the lights after they detect motion, then after a short while turn them off – they are typically used for utility and outdoor security lighting. There are also occupancy sensors used in rooms, which turn on the lights when they detect indoor activity, then turn them off or reduce the light output when the particular space is unoccupied. 

Switching to energy-efficient light fixtures such as CFL or LED bulbs

Lighting costs are a major contributor to the energy bills being footed by the business. What kind of systems do you have set up?

Incandescent bulbs are rapidly being phased out due to their inefficiencies. They work by a wire tungsten filament getting heated until it glows – a process that sees almost 90% of its energy being released as heat, instead of light. In addition, with an average lifespan of just 1,500 hours, there is the need for better alternatives – and they have already been around for over a decade: CFL and LED bulbs, which save on energy and have far less carbon emissions. 

Compact fluorescent light bulbs (CFLs) light up when an electric current going through a tube with argon and trace mercury gases generates ultraviolet light, stimulating the fluorescent coating that’s on the inside of the tube, which in turn produces light. As such, a 15-watt CFL will have about the same light output as a 60-watt incandescent bulb. This makes them approximately 4 times more efficient compared to the incandescent bulbs, with a lifespan of 10,000-15,000 hours. This translates into fewer replacements and greater energy savings. However, there are still concerns about the mercury that is in the CFLs, though it is still in small quantities – basically smaller than the tip of your pencil. In addition, the CFLS aren’t; dimmable. They are usually used as a replacement for incandescent bulbs before completely switching to the more efficient LEDs.

Light-emitting diode bulbs (LEDs) Take things a notch higher. Here, electrons moving through a semiconductor emit the light, and you can get the LEDs for visible light, ultra-violet, and infrared spectrums. Here, the lifespan is 25,000–35,000 hours, which is more than double that of CFLs, and leagues beyond the standard incandescent bulb. Moreover, with a 16.5W LED bulb you’ll be getting the same lighting as a 20W CFL, or a 75W incandescent bulb. 

You will notice that when you touch LEDs, they feel cool, and this is because less energy is getting converted into heat. With the energy efficient bulbs, you won’t have to run your AC harder during those hot months, further adding to your cost savings. You can be able to see such consumption trends over the months through the energy management system, getting to the root cause of the problem. For instance, seeing the changing trends in the AC energy consumption over different weeks will enable you to assess what is causing it to be pushed harder, and address the root cause of the problem. 

Acquiring energy-efficient office equipment

This is broad, with the changes being made here depending on your particular niche. Take printers for instance. Simply going for printers with sleep and automatic shut-off modes will ensure that the units are not consuming energy when they are not in use. The same case applies to copier machines. Energy saving surge protectors on the other hand are beneficial for allowing you to “unplug” multiple devices that use standby power even when switched off – what’s usually called “vampire power” or “phantom energy“. 

The need for energy savings cuts across the board, from the computers and monitors used, to the coffee makers and kettles. For instance, working with an electric kettle to heat water for tea beats using a microwave or stove. Go further by opting for a kettle that allows you to set the particular temperature you want for the water – since you don’t really need the water for tea to be boiling hot for the tea to properly steep. Taking such steps further contributes to your business’ efforts to go green and reduce your carbon footprint. 

Turning to renewable energy sources

Switching to renewable sources to power your operations will simultaneously reduce your energy bills and cut your carbon emissions. From solar panels to wind turbines and the like, they are cleaner sources of energy, and the installations that you go with will depend on your kind of business. Moreover, this will protect you from the fluctuations in energy prices, since the bills are affected by the availability of fuel, electricity demand, costs that go into generating and distributing it – all of which end up hitting your business in the long run. On the other hand, going off the grid with your own supply of power protects you from this. In fact, if you end up producing surplus energy, you can sell it back to the grid, earning your business extra revenue. 

Sure, the upfront costs of setting up the systems will take a sizable chunk out of your budget, but the savings allow you to recoup the costs over time. In addition, there will be savings from the incentives being provided by the government, such as tax rebates and grants. These are the likes of the Solar PV Grant from SEAI (Sustainable Energy Authority of Ireland) which is at €900 per kWp, capped at €2400 for each business. Funding is available for homes, community programs and commercial buildings such as  Collinstown Park School that was able to slash their lighting costs by a whopping 90% after securing 50% of the funding for their energy upgrade project from SEAI. The ecoVaro EMS comes with support for solar power installations in its firmware, that way you can continue assessing the changes that your solar power system will bring to your overall energy usage.

Spread awareness

You should also carry out energy conservation training for your staff. The reports generated by the EMS will make it easy for them to get a picture of their energy consumption trends, and the effects that it has on both the performance of the company, and the carbon footprint as a whole. It also gives them more awareness of the impact that they each have at an individual level. 

Assessing Key Performance Indicators

The energy analytics tools from the EMS will show you whether you are actually meeting your goals. Since it works with the different metered connections, from getting electricity and temperature readings, checking radiation levels, humidity data all through to gas meters, you will be able to assess the progress that your business is making across the board. 

For ecoVaro in particular, the performance of your systems can be seen through reports like Consumption Charts – from the different offices, tenants and equipment energy usage, peak -and off-peak data, as well as Regression Charts that allow you to compare building’s actual energy consumption to its expected performance, and how they are affected by variables such as temperature. 

With the site-by-site data and the monitoring being down to the sub-meter level, you will be able to identify an issue when it crops up and narrow it down to the specific instant and location where it occurred. This enables you to address the problem quicker.   

Conducting a compliance audit

A comprehensive audit can then be undertaken to ensure that your company meets internationally-recognized standards that have been stipulated regarding implementing energy management systems and enhancing the energy efficiency of your operations. The compliance audits are carried out by certified auditors.

Through the EMS, you are able to position your business appropriately to meet the standards for your particular niche, measuring and observing the performance of energy-saving projects that have been implemented. This extends to acquiring and presenting data that will be used to show the business’s compliance to industry regulations and obtain the relevant certification. You are able to report on your carbon footprint, and verify it. This information can also be disseminated amongst your employees and customers, raising awareness about your business green initiatives, boosting your brand in the process.

Energy efficiency demystified

Energy bills are all about Energy efficiency but energy efficiency management is not all about bills. Energy efficiency means reducing carbon emissions, lowering energy costs and improving the quality of life. Energy efficiency is therefore about conservation of energy in a broader perspective; in fact energy efficiency is almost becoming a moral obligation.

Through adoption of appropriate energy efficiency measures, companies can significantly bring down the overhead costs making hundreds of dollars in savings. Energy efficiency is also synonymous with a better quality of life. Taking appropriate measures to ensure proper insulation protects your premises against extreme weather conditions leading to more productivity and an improvement in the bottom line.

Improved energy efficiency means a smaller amount of carbon emissions, less pollution and a better environment.

It is now easier than ever to visually identify where your facility is wasting energy, how much energy is being wasted while tracking the progress made in reducing energy consumption by turning that detailed, raw energy-consumption data into useful charts and figures.

Having visibility of your Energy usage gives you knowledge of what power you are consuming. This helps you change energy usage behaviours and this can have significant savings and reduction in your electricity bills. Real-time electricity consumption tracking is enough prodding for you to be on the lookout for inefficient energy consumption unit’s e.g.? Heating and cooling equipment, ducts insulation of your premises or a failure of one of these components to perform as intended. Pin-pointing the problem areas is not a walk in the park but fixing it can make your building more energy-efficient and comfortable.

A wide range of solutions are now available for charting and analysing energy consumption that helps energy managers, facilities managers, energy consultants and building-services engineers. These will not only offer advice but will enable you provide tailor made solutions for your organisation by assisting you in developing a sustainable energy strategy. Our energy monitoring software is one example.?

How Internal Auditors can win The War against Spreadsheet Fraud

To prevent another round of million dollar scandals due to fraudulent manipulations on spreadsheets, regulatory bodies have launched major offensives against these well-loved User Developed Applications (UDAs). Naturally, internal auditors are front and center in carrying out these offensives.

While regulations like the Sarbanes-Oxley Act, Dodd-Frank Act, and Solvency II can only be effective if end users are able to carry out the activities and practices required of them, auditors need to ascertain that they have. Sad to say, when it comes to spreadsheets, that is easier said than done.

Because spreadsheets are loosely distributed by nature, internal auditors always find it hard to: locate them, identify ownership, and trace their relationships with other spreadsheets. Now, we’re still talking about naturally occurring spreadsheets. How much more with files that have been deliberately tampered?

Spreadsheets can be altered in a variety of ways, especially if the purpose is to conceal fraudulent activities. Fraudsters can, for instance:

  • hide columns or rows,
  • perform conditional formatting, which changes the appearance of cells depending on certain values
  • replace cell entries with false values either through direct input or by linking to other spreadsheet sources
  • apply small, incremental changes in multiple cells or even spreadsheets to avoid detection
  • design macros and user defined functions to carry out fraudulent manipulations automatically

Recognising the seemingly insurmountable task ahead, the Institute of Internal Auditors released a guide designed specifically for the task of auditing user-developed applications, which of course includes spreadsheets.

But is this really the weapon internal auditors should be wielding in their quest to bring down spreadsheet fraud? Our answer is no. In fact, we believe no such weapon has to be wielded at all?because the only way to get rid of spreadsheet fraud is to eliminate spreadsheets once and for all.

Imagine how easy it would be for internal auditors to conduct their audits if data were kept in a centralised server instead of being scattered throughout the organisation in end-user hard drives.

And that’s not all. Because a server-based solution can be configured to have its own built-in controls, all your data will be under lock and key; unlike spreadsheet-based systems wherein storing a spreadsheet file inside a password-protected workstation does not guarantee equal security for all the other spreadsheets scattered throughout your company.

Learn more about Denizon’s server application solutions and discover a more efficient way for your internal auditors to carry out their jobs.

More Spreadsheet Blogs

 

Spreadsheet Risks in Banks

 

Top 10 Disadvantages of Spreadsheets

 

Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry

 

How Internal Auditors can win the War against Spreadsheet Fraud

 

Spreadsheet Reporting – No Room in your company in an age of Business Intelligence

 

Still looking for a Way to Consolidate Excel Spreadsheets?

 

Disadvantages of Spreadsheets

 

Spreadsheet woes – ill equipped for an Agile Business Environment

 

Spreadsheet Fraud

 

Spreadsheet Woes – Limited features for easy adoption of a control framework

 

Spreadsheet woes – Burden in SOX Compliance and other Regulations

 

Spreadsheet Risk Issues

 

Server Application Solutions – Don’t let Spreadsheets hold your Business back

 

Why Spreadsheets can send the pillars of Solvency II crashing down

?

Advert-Book-UK

amazon.co.uk

?

Advert-Book-USA

amazon.com

 

Ready to work with Denizon?