"ERP: Ten Years and Fifty Million Dollars"
"ERP: Fifty-Eight Percent Failure Rate"
"ERP: Multi-Year Project Scrapped - IT OutSourced"
Industry trade articles and leading industry analysts have been headlining the challenges and failures companies worldwide have been facing when trying to implement major enterprise applications such as Enterprise Resource Planning - ERP.
The symptoms are familiar: order-of-magnitude budget overruns; career-spanning implementation timelines; cross-division political bickering; application functionality shortfalls; endless scope creep; inadequate or brittle infrastructure.
The problems are real. The causes varied and complex: lack of executive sponsorship; business-IT misalignment; inadequate program management; application functionality shortfalls; brittle support processes, non-scalable infrastructure. Fortunately, many good consulting firms are ready and able to assist with these issues. From a business perspective, however, few projects have a higher total cost of ownership than failed ERP projects and return on investment in these circumstances is limited to the painful-to-swallow "lessons learned".
One area of ERP that can have a significant impact on total cost of ownership and return on investment is 'infrastructure'. Typically, however, infrastructure issues are thought to be 'technical' and are left to evangelization by aggressive hardware sales reps and IT technicians touting the latest 'giga-cache bio-fusion units' and 'diabolical system doo-hickeys'. Technical analyses in the guise of benchmarks, vendor 'bake-offs", or 'techno-feature beauty contests' are often substituted for business analysis. But infrastructure issues yield quite readily to thoughtful business and financial analysis once fundamental ERP requirements and hardware vendors' hardware architectures are understood.
What are the characteristics that all major ERP packages have in common? What type of physical infrastructure is best suited for ERP? Are all hardware architectures suited equally well? Can there be mismatches between ERP software architecture and hardware architectures? What effect would these mismatches have on TCO and ROI?
This goal of this paper is simple: to explain the potential TCO and ROI implications of mismatches between ERP software architecture and the leading hardware architectures being sold for ERP environments. Armed with this knowledge, IT and senior business executives can make more informed decisions for the life of their business.
This paper is divided into eight sections:
ERP Infrastructure:
The set of components required to support ERP processing.
- physical components (servers, networks, storage, clients)
- people (business staff, operations staff, development staff)
- organizational processes (program and project management, change management, support services, etc.)Each of these infrastructure components is critical to ERP success. The focus of this paper, however, will primarily be on the physical infrastructure components and how decisions about these physical components for ERP can affect TCO and ROI.
TCO:
Total Cost of Ownership; all cost incurred over ERP's lifetime.
This concept, popularized several years ago by industry consultants, focuses on cost, not value. Nevertheless, a true TCO analysis can be useful if applied carefully to the specific type of environment being implemented. Consultants typically speak about TCO over a fairly limited span - i.e. "Five Year TCO". Although it is almost impossible to accurately predict 'total cost' over the 20 or 30 years of an ERP lifespan, the popular "Five Year Total Cost of Ownership" can have a crack so large that literally hundreds of thousands of ERP infrastructure dollars fall through unforeseen.
A Five Year TCO analysis is based on the following assumptions:
- costs will be evenly spread across all five year periods
- all hardware architectures for ERP are roughly equivalent in their ability and their cost to accommodate changing business requirements as well as growth in ERP and bolt-on ERP applications such as DataWarehousing, eCommerce, Supply Chain Planning, etc.
We will see later that when ERP's architectural requirements and multi-year implementation timelines are compared to hardware architectures for ERP, that these two 'garden-path' assumptions can have unexpected budget-busting ramifications.
ROI:
Return on InvestmentReturn on Investment has both a "how much" and a "when" element. The "how much" represents the actual dollars generated by the investment. The "when" represents how soon a return can be realized.
Certainly the amount of the return will vary from company to company, from implementation to implementation, from business situation to business situation. ERP infrastructure, on the other hand, can definitely affect the when - the "how soon" any return can be achieved. If a business objective can be achieved with either fewer dollars or with the same number of dollars spent later then an earlier return on investment can be achieved.
This paper primarily focuses on the "when" component of ROI but will also give some insights into lowering overall expenditures for ERP infrastructure. When comparing ERP hardware architectures with ERP software architectural requirements we will look to identify opportunities to defer acquisitions and reduce infrastructure cost yet provide equal or superior scalability and investment protection.
Nodes/Domains
Refers to physical server hardware
Terms used interchangeably. A 'node' or 'domain' is a unit of hardware (some number of CPUs) that runs a single operating system (OS) image.
Example: Four groups of CPUs running eight copies or versions of an OS would be referred to as four domains or nodes.
ERP, Enterprise Resource Planning, is an outgrowth and extension of MRP, Manufacturing Resource Planning. MRP, later referred to as MRP II, was a collection of inter-related information processing modules that coordinated and guided materials and resources - from input to finished product output - through the manufacturing process. The goal of MRP was to speed time-to-market and decrease inventory and manufacturing costs by optimizing schedules and paths of materials and resources through the manufacturing process. With process standardization, MRP also provided a good basis for quality measurement and continuous improvement initiatives.
In a similar fashion, ERP can be thought of as the measured flow and transformation of information through the business processes of an organization. ERP then, can be viewed as a 'software mirror image' of the major business processes of an organization.
ERP makes two key contributions to business:
With ERP business processes share the same definition of data across all ERP
application modules. This is in stark contrast to pre-ERP days when multiple
standalone applications not only used their own copy of data but data elements
were not defined in a standardized way across business processes.
A basic design objective of ERP is that a single set of data be maintained across all business processes within an organization. Before ERP, multiple copies of data were maintained across an organization processed by multiple standalone applications. Under these circumstances gaining an understanding of the business health of the company was a guessing game at best. Vital business decisions were being based on faulty, inaccurate, non-normalized data. Executive decision making was often more an art than a science.
From a business-process perspective, where is quality maintained and where is value created (or lost) with ERP? Quality comes at each processing step - i.e. how well does each ERP software module model actual business processes? Value (speed, agility, flexibility, scalability) comes at the links between the steps.
An example:
A major university measures its 'capture rate'- the percentage of students who actually enroll in the university after having been accepted for admission. The school administration has determined that on average each of its student applicants also applies to five other universities.
They have measured that 40% of students who apply are accepted and 65% of those accepted enroll. So, the 'capture rate' is a good measure of this school's competitiveness in enrolling students. What are the factors that contribute to a high capture rate? How could a good ERP package be of value to the university in winning students (customers)?
ERP, conceptually defined, is a software mirror image of an organization's business processes. Although a university has no manufacturing, there are a series of coordinated business processes that must be linked together for the university to produce business results - i.e. enroll students. Viewed end-to-end, the business processes look similar to this:
This university believes that time is a significant contributor to the capture rate. Other factors being roughly equal (amount of financial aid, etc.) applicants tend to enroll at the first school that responds to them - before they have an opportunity to emotionally or financially commit to other alternatives. If the university can return positive admission and financial aid determinations to the applicant quickly then the capture rate goes up.
So 'quality' (financial aid, academic review, etc.) is determined at each processing step (application module) and 'value' is determined by the speed of moving through the business processes. In this respect, the parallels to manufacturing are clear.
It is also easy to see that ERP infrastructure can play a significant role in value. An unreliable, unscalable, 'slow' ERP infrastructure could clearly bog down information flow through the ERP business processes and thus increase the time for the university to return a determination. This could have a severe impact on the capture rate. Conversely (at least to a certain point), the faster the information flows through the business processes, the better the 'capture rate' for the university.
Multi-Tier Software Design
Current generation ERP software packages are architected as multi-tier applications using three or more tiers. Each tier is implemented in one or more nodes or domains. Each node or domain has its own OS (operating system) image. Internet web-based client access has certainly accelerated this "tiered" trend.
Typically, the database engine resides at one tier, a transaction monitor may occupy tier two, multiple application modules reside in one or more nodes or domains at a third tier, network security at a fourth, client access at a fifth.
A multi-tier software architecture provides many advantages. For the ERP software vendor, this architecture provides the opportunity to:
For the ERP customer, a multi-tier architecture provides the opportunity to:
Implementing ERP functionality and infrastructure in a multi-tier component fashion could significantly decrease 'switching costs' incurred when moving to a new environment as modular replacements can be made rather more quickly and less expensively than an 'all or nothing' complete replacement.
Hardware Architecture Implications
This high speed interconnect should be reserved for tier-to-tier and inter- application communication only. In order to maintain both throughput and security, this high speed interconnect should not be used for user access. In order to avoid slowing down both inter-tier communication as well as 'within-a-node' processing, it should not be used for normal node or domain operations such as CPU to memory and I/O to memory access.
Hardware vendors call their implementation of this high speed interconnect by various names. Capabilities and speeds can vary substantially. Properly designed and implemented, this high speed interconnect should not interfere with regular intra-node or intra-domain processing nor should it bottleneck inter-tier or inter-node processing. A high-speed, low-latency interconnect can also be important for other uses:
The best tools for managing an ERP environment provide the business-technology linkages for the IT operations staff. These tools allow the operations staff to manage the technology from a business perspective.
These tools should be pre-tested and pre-engineered to work with the ERP software environment that is being implemented (Oracle, SAP, Baan, Lawson, PeopleSoft, etc.). Custom-integration of management tools adds time and cost to an ERP implementation. The best infrastructure vendors will have already pre-engineered and pre-tested a set of IT management tools that work with your chosen ERP software environment.
Most vendors design the modules of ERP to run either standalone or as an integrated set. This design allows vendors to sell complete ERP suites or just individual modules and allows companies to implement and digest ERP in bite-sized pieces.
Generally speaking, each module has its own performance characteristics, related to, but independent of the performance characteristics of the other modules. As the number of users, employees, customers, products, sales, and transactions grow - the power requirements of each application module tends to grow. But, the processing power growth rate is not necessarily the same for all modules.
For example, hiring 50 new employees in a 10,000 employee company would require additional data storage for the new employee records but would not require much additional processing power. Adding 50 new products to the manufacturing line, however, would require significant growth in processing power to take care of material handling, production and employing scheduling, supplier relationships, accounts payable, etc.
![[]](IMAGE007.GIF)
More importantly, the inverse is not true. It is not true that an inverse relationship necessarily exists between the processing power requirements of various modules. If processing power requirements of one or more modules increase (due to growth of business conditions) it is highly unlikely that any other modules will experience a corresponding decrease in processing power requirements. How application modules utilize hardware resources can be addressed by your chosen ERP software vendor. While sizing hardware for ERP software can sometimes be more of an art than a science ERP software vendors partner with the leading hardware vendors to do hardware sizing and configurations for each ERP module.
Hardware Architecture Implications
ERP Characteristic: Multi-year Implementation
Most organizations implement complex ERP packages a module or two at a time over a period of several years. Industry trade reports observe that some multi-division implementations have taken as long as 5 to 10 years. Why are implementation cycles so long?
To be effective and to have value, ERP software must model a company's actual business processes. Either business processes must be changed to fit the ERP software or ERP software must be adapted to the changing business processes required over time in a competitive market. ERP, then, clearly requires a strategic business commitment and cannot be implemented overnight.
What amount of investment up-front must be made in hardware infrastructure in order to launch ERP? Should an organization buy the 'biggest' system possible or start small? How should a company get started implementing ERP without over-purchasing hardware that won't be used for many months or even years? Time is central to the analysis of TCO and ROI for ERP infrastructure. There can be significant TCO and ROI implications when forced to buy hardware too early. We shall illustrate the answers to these questions in the "Financial Analysis" section later in this paper.
Hardware Architecture Implications
ERP should not be viewed as a 'project' but rather as a business-lifetime commitment. ERP is a software-mirror image of a company's business processes. As an increasing number of business transactions and processes move to the Internet, many are predicting that, in essence, an extended ERP will become your business.
With this type of business lifetime commitment, how does a company avoid vendor 'lock-in'? What can be done to ensure that the substantial financial outlay for today's 'latest and greatest' ERP physical infrastructure does not become tomorrow's 'golden handcuffs'? How to avoid being tied to uncompetitive support pricing or to hardware and an operating system that in the future will be considered prohibitively expensive, inflexible, and obsolete?
Certainly business conditions change, often unexpectedly. The special challenge of building an ERP environment is that infrastructure investments made today will have to accommodate business condition changes years into the future: mergers, acquisitions, divestitures, spin-offs, and even new business paradigms (like the Internet). These types of changes place profound demands upon a company's ERP infrastructure.
Hardware Architecture Implications
ERP Characteristic: Grows Large and Then Grows Larger Still
The past several years have seen an explosion not only in data storage requirements but also in processing power requirements for large organizations. ERP has been responsible for much of this growth. Business computing power is often measured in "transactions per minute" or "TPM". It is not unusual for large organizations to require greater than 200,000 TPMs for their total ERP environment including test/development, promote-to-production, production, High Availability, and Disaster Recovery. Even if all ERP processing could be performed in a single node or domain the largest server node available today is in the 120,000 TPM range.
Hardware Architecture Implications
ERP is the very definition of 'mission-critical'. Uptime and data protection is vitally important to every business. For large organizations, direct downtime costs begin in the tens of thousands of dollars per hour and increase from there. And direct costs do not take into account indirect costs such as the loss of customer loyalty and goodwill or even the ability of the business to survive. The overall cost and impact can be staggering. It has been said that in the aftermath of the New York World Trade Center bombing 40% of businesses who lost access to core ERP data for 24 hours or more were out of business within one year. But what if calculating the cost of downtime in your current environment is too difficult? Perhaps a qualitative analysis will suffice.
Many organizations choose to implement ERP to get tighter control of their business processes. When coming from a pre-ERP environment it can be difficult to calculate the real dollar cost of downtime because often adequate control and measurement mechanisms are not in place in the current environment. Yet it is intuitive that some level of infrastructure investment must be made to protect availability of the new ERP environment. Ramneek Bhasin, Executive Vice-President of WSB Technologies of Wolfeboro, N.H. speaks of the concept of a 'lifeboat drill'. The lifeboat is your finite IT budget. Which applications, data, and systems must you take into the lifeboat in order to keep your company afloat? The question then quickly becomes, 'Can your business survive without access to core ERP processing and data?'
Hardware Architecture Implications
ERP Characteristics: Multiple Environments Required
For ERP, most businesses must maintain multiple environments in order to achieve uptime requirements: development/test, promote-to-production, highly-available production, and disaster-recovery. In fact, some companies maintain as many as seven distinct environments. So to provide the necessary infrastructure for ERP (or in fact for any major enterprise application) the financial stakes are even more daunting and the need to engineer infrastructure acquisition and deployment in order to hold down TCO and speed ROI is even more important.
The good news is that some of these environments can perform 'double-duty' particularly with regards to high availability and scalability. Many companies use parts of their development and promote-to-production environments to provide high-availability failover and peak-demand scalability for the production environment. The key is to engineer an infrastructure that provides the right balance between cost and availability, and between development, promote-to-production, production, and disaster-recovery environments.
Hardware Architecture Implications
System hardware vendors inexperienced with mission-critical datacenter environments sometimes imply that their hardware can be reconfigured 'dynamically' to meet changing production requirements. But be careful about the word 'dynamic'. 'Dynamic' implies automatic adjustment in response to changing circumstances. Vendors who market 'dynamic' but are really offering 're-configurable' or 'it-can-be-changed' are probably not very experienced with mission-critical datacenter environments.
True 'dynamic' reconfiguration is rarely the case with computing hardware except with specialized switching hardware for networks and telecommunications or with very expensive active-redundant, non-stop hardware such as from Tandem or Pyramid. When some vendors tout 'dynamic' - they really mean 're-configurable' or 'changeable'.
There are two main alternate approaches to 'flexibility' relative to hardware architectures. One approach is to switch hardware components from one node or domain to another within a server to accommodate changing requirements. The other approach is to migrate software between the nodes or domains within the server to adjust to changing requirements. While neither of these approaches is 'dynamic' there are significant architectural differences between the two.
The hardware-change approach typically requires more complicated and more expensive hardware. Typically, to qualify for support, analysis must be performed by vendor hardware engineers several weeks before the proposed change. This is to insure that no negative performance consequences result from the proposed change. Done improperly, these types of changes can result in an unbalanced system with CPU, memory, or I/O bottlenecks.
The software-change approach relies on software rather than on specialized hardware. Remember that the goal in designing reliable hardware is to reduce the number of active components and to reduce complexity. From a hardware reliability perspective, then, the software approach using less complex hardware should be more reliable.
While both approaches require that changes be pre-planned, it is important to realize that the two approaches test change at different times.
The software approach relies on control scripts that direct the starting, shutdown, and movement of application executables for load balancing and failover. These control scripts are pre-tested prior to production and prior to change events.
The hardware approach, on the other hand, relies on specialized hardware circuitry in the internal system interconnect which is exercised only during change events. This means that with the hardware approach testing effectively happens in real-time - exactly when you need the change to occur. If the specialized hardware circuitry is to fail, it will fail exactly when you need it not to fail - at the time of the change event.
For the software approach, look for a vendor whose tools allow you to:
ERP Trends: Component Mix and Match, Best of Breed
The original promise from ERP software vendors was that a single ERP package from a single vendor offered the best integration of component modules, the best integration of business processes and the tightest control of common data definitions (metadata). Over the past several years, however, (for a variety of reasons) a number of trends began to emerge:
So pronounced have these trends been that a new breed of ERP middleware has emerged in the marketplace known as EAI - Enterprise Application Integration. With this trend, ERP integration issues have moved to the forefront of ERP consulting activities.
Hardware Architecture Implications
If the OS vendor releases versions too quickly (more than one every 18 months or so) then it can drive up cost-of-operations and complicate application-OS dependencies. Faster OS version release frequencies can actually slow down new application release availability for a particular production environment as software vendors typically release their latest application version on the latest OS version. If the OS vendor releases new versions too quickly then there is a good chance that your production environment will be running an older OS release. (Also, if the OS vendor releases new versions too quickly there is a very good chance that the OS is not of mission-critical production quality). Application-to-OS, domain-to-domain, or OS-to-management-software release dependencies may prevent you from adopting new application functionality which in turn can affect how soon desired business value can be delivered to the organization.
ERP Trends: 'Bolt-On' ERP Add-ons
There is a growing list of popular bolt-on ERP applications that consume cycles and
gigabytes of storage quite readily: Data Warehousing, Supply Chain Management, Customer Relationship Management, web-based procurement and others. Each of these can share data with core ERP modules and taken together with ERP consume an ever-increasing number of CPU cycles and gigabytes of storage.
We have discussed multiple ERP characteristics and trends. Taken together, these characteristics and trends dictate five fundamental architectural requirements for all modern ERP environments:
Requirement #1:
ERP modules require common access to data
Requirement #2:
Multiple OS images or domains are required to
host the multiple tiers and components of the ERP software.
Requirement #3:
Fast, efficient inter-module, inter-tier
communication is required.
Requirement #4:
Single Point of Management.
Common management of the different tiers, OS images, and application
components must be used to decrease operations management costs and to help
ensure coordinated management of ERP business processes.
Requirement #5:
Robust, Secure Client Access
![[]](IMAGE009.GIF)
These fundamental ERP architectural requirements influence how vendors build servers
for ERP. Each of the major hardware vendors architect 'clusters' to meet these requirements but do so in very different ways. First, we'll examine the definition of 'cluster' and then identify the two main types of cluster architectures used for ERP.
Clusters
All major vendors support the requirement for multiple domains or OS images with various implementations of "loosely-coupled clusters". "Cluster" simply implies multiple OS images or 'domains' connected together via some type of cross-connection (usually a network connection). "Loosely-coupled" simply means that each OS image or domain has its own private memory space. Private memory space is very desirable since it isolates each application module from the others and thus protects a "rogue" or "ill-behaving" module from directly affecting the others.
Two Types of Cluster Architectures for ERP
Each of the three major hardware vendors (IBM, Sun, HP) provide "clusters" for ERP.
There are two types of cluster architectures for "ERP Clusters":
These two cluster architectures differ in the placement of exterior sheet metal and the location of the node or domain inter-connect. We shall see that the location of the 'skins' and the location of the high-speed interconnect are the critical factors in scalability and how smooth or 'spiky' incremental costs are for expansion and scalability, and performance. A more detailed look at these two cluster architectures is presented on the following two pages.
![[]](IMAGE010.GIF)
| ||||||||||||||||||||||||||||||||||||||||||
"Inside-the-Skins" designs:
![[]](IMAGE011.GIF)
| |||||||||||||||||||||||||||||||||||||
"Outside-the-Skins" design:
Distinguishing Characteristics of Cluster Architectures for ERP
As you can see from the above diagrams and tables both types of architectures utilize a switched interconnect to connect the nodes or domains. There are, however, two key architectural differences:
For the "Inside-the-Skins" architecture, the scalability limit is the physical skins - the frame itself. Frame proponents suggest that multiple frames can be used so the number of domains can actually be higher than what can fit inside a single frame. While this is possible, all of the attractive "Inside-the-Skins" hardware flexibility features become problematic - as they only work "inside the skins" - not "between the skins". When we look at the ERP infrastructure financial analysis in the last section we shall see that adding multiple frames in an environment can have a very significant effect on TCO and ROI.
For the "Outside-the-Skins" architecture, the scalability limit on the number of domains is defined by the external switched interconnect (for HP, the 'HyperFabric Switch').
- "Inside-the-Skins" - the interconnect is - inside the skins (inside the frame).
- "Outside-the-Skins" - the interconnect is outside the skins, the external switch.
You may be beginning to suspect that, given the growth and scalability requirements of ERP and bolt-on ERP applications, the greatest strength of the "Inside-the-Skins" architecture may also be, for ERP, its greatest weakness: the skins (the frame). It is easy to see that any potential flexibility derived from containing multiple domains within the skins could be overwhelmed by the scalability limitations imposed by those same skins. Perhaps Cole Porter, the great American songwriter, had ERP environments in mind when he wrote his famous western song, "Don't Fence Me In".
It is important to point out that any disadvantages an "Inside-the-Skins" architecture may have for ERP do not necessarily carry over to all other application environments. In fact, an "Inside-the-Skins" architecture can be quite attractive for certain types of environments:
It was, in fact, for these types of environments that "Inside-the-Skins" systems were originally architected.
Now that we have looked at both the architectures of ERP software and the architecture of the two types of hardware offered for ERP environments, we can return to the central question:
To understand the financial implications of "Inside-the-Skins" versus "Outside-the-Skins" architectures for ERP we will look at four different points during the lifecycle of ERP. It is particularly at the last three points in the lifecycle where the typical "5 Year TCO" analysis may allow hundreds of thousands of infrastructure dollars to "fall through the cracks" unforeseen:
These four points in an ERP lifecycle represent times of significant change. With regards to infrastructure, change costs money. Depending on the hardware architecture chosen for ERP these times of change can induce moderate costs or very large costs. Let's compare the financial implications of each architecture at each of these times of change during an ERP lifecycle.
Lifecycle Time #1: ERP-Startup
ERP projects are multi-million dollar projects. Budgeting for ERP projects can be similar to walking into a new car showroom for the first time in 30 years. Severe sticker shock. Times an order of magnitude. Or two. The costs are significant:
Anything that can be done to lessen the first year or 18 months of financial outlays without jeopardizing project success would be very desirable. This is the first point in time where the choice of hardware architecture for ERP will make a difference.
The fundamental 'flexibility' premise of an "Inside-the-Skins" architecture is that system resources will be able to be shared between parts of the overall environment. But we have seen in a production ERP environment that there is not an inverse relationship for system resource utilization between modules or tiers. That is, there is no guarantee that the resources required during production by any one particular module will decrease at the same time and by the same amount and for the same length of time that another module's resource requirements increase. This means that in order for the basic "Inside-the-Skins" flexibility premise to hold true for ERP - that the flexibility of resource utilization must come between the production environment and the development, quality assurance, and training environments.
This has multiple implications:
Let's compare "Inside-the-Skins" vs. "Outside-the-Skins" architectures for ERP. For purposes of this comparison ERP hardware from Sun Microsystems and Hewlett-Packard will be used as examples.
Sun markets an "Inside-the-Skins" architecture with their UE10000 "Starfire" server. Sun also sells standalone nodes that can be clustered. Conversely, Hewlett-Packard does not market an "Inside-the-Skins" server for ERP. Rather, HP has architected an "Outside-the-Skins" server for ERP called 'HP HyperPlex'.
To drive home the comparison and avoid "religious" vendor comparisons a number of server infrastructure comparisons will be made:
A scalability table for each of the nodes is presented below. For this comparison a 'relative processing power index' (RPPI) is used. Another source of performance comparison are TPC ratings from the Transaction Processing Council (TPC). TPC provides standard, audited transaction-oriented TPC-C benchmarks. Their ratings can be found at www.tpc.org. TPC-C is a benchmark certified by the Transaction Processing Council for transaction environments. TPC-D is a benchmark that focuses on decision-support or data warehouse type environments. Since ERP is a transaction-oriented environment, TPC-C ratings are a reasonable indicator of performance for ERP.
| Node Type (fully configured) |
Relative Processing Power Index |
Max CPUs | Max Memory | Backplane |
| Sun UE 10000 | 11 - 12 | 64 | 64 GB | 12.8 GB/sec |
| Sun UE 6500 | 5.1 - 5.4 | 30 | 30 GB | 3.26 GB/sec |
| HP N 4000 | 4.8 - 5.0 | 8 | 16 GB | 7.6 GB/sec mem 3.8 GB/sec sys |
| Sun UE 5500 | 3.6 - 4.0 | 14 | 14 GB | 3.26 GB/sec |
| Sun UE 3500 | 2.5 - 3.0 | 8 | 8 GB | 3.26 GB/sec |
![[]](IMAGE013.GIF)
A Comparison
"Inside-the-Skins" systems have a basic starting requirement: before you purchase any compute power - you must first purchase the "skins", i.e. the 'frame'. For the Sun "Inside-the-Skins" system, the UE10000, the entry-level requirements are the frame along with a 'control board', 4 'system boards' and 16 processors. Let's assume that, taken together these 16 processors will produce a relative processing power index (RPPI) in the range of 4 - 6. The Sun system can be divided into 'domains' - so let's assume four domains of four CPUs each.
For the "Outside-the-Skins" configurations, we shall normalize to four nodes or domains to match the UE10000 "Inside-the-Skins" configuration. An aggregate RPPI rating of these four partially-configured nodes will fall approximately in the same range - about 6.
Costs are shown below for these "Entry-level ERP environment" systems. For ease of comparison, CPU hardware costs are compared. Memory and I/O controller costs are not included because pricing for these items is relatively standard across vendors and would not influence comparative costs. At least in the case of the HP N4000, the operating system comes free with the system - so no additional charge is added for OS for either vendor on any of the systems. Additionally, because support pricing is typically just a percentage of hardware pricing, we will also ignore support pricing. (By ignoring support pricing, a higher-priced system might be getting an 'unfair' break. For purposes of this analysis we'll just call support pricing more or less even as well.)
We shall see in the following graph there is a significant cost penalty to be paid, regardless of vendor, in using an "Inside-the-Skins" architecture for ERP.
![[]](IMAGE015.GIF)
Configuration Note:
Sun UE10000 configured with minimum allowable CPU configuration: 16 CPUs
Sun UE6500/5500/3500 (qty 4). Each configured with 4 CPUs. (aggregate RPPI ~ 6)
HP N4000 (qty4). Each configured with 2 CPUs. (aggregate RPPI ~ 6)
HP N4000 HyperPlex. Four N4000 nodes with dual, redundant HyperFabric switches.
So, what does this graph show? Some observations:
- 60% more expensive than a UE5500 "Outside-the-Skins" cluster
- 136% more expensive than a UE 3500 "Outside-the-Skins" cluster
- 172% more expensive than an HP N4000 "Outside-the-Skins" cluster
- 105% more expensive than an HP N4000 HyperPlex
Opportunity Borrowed from the Bottom Line
It is not uncommon for the initial development periods for medium or large-sized ERP
implementations to be nine to fifteen months, or even greater. As noted earlier, purchasing physical infrastructure long before it returns value can significantly delay the point in time where ROI is achieved. Not only is the initial infrastructure more expensive but these are nearly a half-million business dollars that would otherwise fall right to the bottom line or could be spent in other business areas where return on investment can be achieved more readily. When viewed from this perspective, companies are paying twice for an "Inside-the-Skins" architecture: once for the increased purchase price and second for the lost business opportunity to invest in other areas of business need.
Lifecycle Time #2: Throughput Growth
Another point in the lifecycle of ERP environments for large implementations comes
when the total aggregate amount of processing power exceeds the total available within an "Inside-the-Skins" frame. This point in time can actually come pretty quickly if both the production-support (development, test, QA, training, etc) and production environments are hosted "Inside-the-Skins". Indeed, it is not uncommon for just the pure production requirements to be in the 200,000 aggregate TPM range. In this situation, how does one provide more processing power to the ERP environment? Depending on the hardware architecture chosen, the answer varies. For each architecture, there are several options.
"Inside-the-Skins" Options:
Here, there can be a difference between "Inside-the-Skins" implementations. The key differentiating factor is whether or not all the nodes or domains share the same 'system clock'. For some implementations (such as the IBM RS6000 SP), each node or domain has its own system clock (i.e. MHz rating). For other implementations (such as the Sun UE10000), all CPUs and domains share a single system clock (i.e. must run at the same MHz rate).
For those systems where all CPUs and domains share a single clock - if an upgrade is
possible - all CPUs must be upgraded at the same time. So, for a CPU-bottlenecked system that has 64 processors, all 64 CPUs must be replaced at once. Because situations can vary so widely and pricing varies from vendor to vendor it is nearly impossible to make a general statement about cost-effectiveness of such an upgrade as compared to adding a new node or upgrading a small number of CPUs in a single domain for an "Outside-the-Skins" architecture.
One business reality, however, is clearly understood by frame vendors: the competition for your business in this mission-critical ERP environment is most likely not another vendor. The 'competition' is the following formula:
(Cost of the new technology) PLUS
(Business Cost of downtime to your business for the entire environment.
Remember, the frame is full so there is no flexibility 'left' to move processing between domains to avoid downtime) PLUS
(Business Cost of Downtime Risk to re-adjust the allocation of processors
between domains in order to provide the CPU cycles to the domain(s) where they are needed (and not to waste the new CPU cycles on the domains where they are not needed.) PLUS
(Business Cost of IT staff time to plan and schedule outages and changes
to the physical environment)
Clearly this situation, even if not prohibitively expensive, can cause great upset to an operational ERP environment and to the services that the ERP environment is delivering to the business and its customers.
From a financial perspective this again induces significantly greater cost than adding the appropriate sized node as one would with an "Outside-the-Skins" solution. In fact, the economic penalty paid at the time of "ERP-Startup" would have to paid again - thus increasing TCO.
This 'hybrid' would consist of the large 'frame' system and one or more smaller nodes (a 'la "Outside-the-Skins"). This option also has problems. First, any in-frame flexibility sought by having all domains "Inside-the-skins" is diminished or lost as you are now building domains "Outside-the-Skins". So to regain this flexibility, you must rely on the 'software flexibility' referred to earlier used by the "Outside-the-Skins" implementations. There is certainly nothing wrong with the 'software flexibility' approach but it begs the question: "Why spend extra money for an "Inside-the-Skins" architecture in the first place? Secondly, going down this path, you have really end up with a "part-Inside / part-Outside" architecture that would only complicate operations.
"Outside-the-Skins" Options:
This is accomplished by normal methods: adding processors, memory, I/O controllers -or- by adding faster processors. (Obviously choosing nodes from a vendor that has a good "internal" upgrade path for each node would be important.) As we can see by referring back to the ERP-Startup analysis, this would certainly be a lower cost than having to purchase a second new "Inside-the-Skins" frame.
If the increased processing power was required at the application tier, adding a node could provide both processing power as well as application-level redundancy for increased high availability. If the increased processing power was required at the database tier (typically the tier requiring the most 'horsepower') the database node would be upgraded to a larger node taking advantage of any vendor financial upgrade plans.
Referring back to the ERP-Startup analysis again, this would still be significantly less expensive than purchasing a new 'frame'. In the event that the node requiring more power outgrew the mid-range nodes being considered (the UE6500 and the HP N-class) then this node would need to be upgraded to the Sun high-end (for a Sun environment) or the HP high-end (in an HP environment).
In the case of Sun, the financial result would be exactly the same as adding a new frame (the UE10000 is Sun's high-end). So, financially, you would not be any worse than in the "Inside-the-Skins" situation. You would have one advantage, however. In this scenario you would not have a 'hybrid' environment because you would be running the UE10000 as a single unified domain in order to get the processing power required for this large node.
In the case of an HP implementation, there would be significant savings as HP's largest node, the HP V-class - not discussed in any detail in this paper, scales nearly as large as the Sun UE10000 but at about half the price.
It is appears then that an "Inside-the-Skins" architecture will be no worse financially than the "Inside-the-Skins" solution and probably much, much better.
Lifecycle Time #3: Domain Growth
Domain growth refers to the situation where the overall processing power of any particular domain does not need to increase but rather a new domain must be added to support another application or middleware module. Let's again look "Inside" and "Outside":
"Inside-the-Skins":
There are really two possibilities: "more domains available" and "no more domains available".
You have not used all the available domains inside the frame and can simply define more (up to the architecture's limit). This obviously would be the happy and the hoped for situation. It does, however, have some long-term disadvantages that drive up total cost of ownership.
On one hand, the longer your frame does not 'fill up' with either domains or CPUs the longer the life span your frame will have. On the other hand, the longer the frame goes unfilled, when you do add CPUs the more you will be paying relative to the then-current price-performance curve of the marketplace. Buying a frame that will not be filled for a time only pushes a company further and further off the industry price-performance curve because the components that go into the frame must be of the same generation as the frame. And in the computing industry, a generation is approximately 2 years. The longer a frame goes unfilled - the worse the economics, the higher the opportunity cost, the higher the TCO.
To paraphrase Vanna White, 'buy a frame'. This is again similar to the ERP-Startup scenario. TCO rises. See above.
Outside-the-Skins
Purchase an additional server node (domain) of the size required. In this situation, it is clear that the "Outside" solution would be much less expensive than buying another new frame. But what happens if the frame is not yet filled? What is more expensive - adding processors, control boards, memory boards, and I/O boards to the frame or buying new small, medium, or large "Outside-the-Skins" server nodes? There are too many variables to consider concerning time and how many extra domains are needed. So, perhaps the only reasonable way to answer this question would be to consider cost for a 'full frame' versus an equivalently sized "Outside-the-Skins" implementation. Please see the graph below.
![[]](IMAGE017.GIF)
From this chart it is clear that the hardware costs significantly favor an "Outside-the-Skins" architecture. When combined with the granular scalability that the "Outside" architecture brings it is difficult to see how an "Inside-the-Skins" architecture could be financially advantageous for ERP - or financially advantageous to your business. To completely identify these cost differences, the author suggests doing a full comparison of total cost (hardware, software, and support). Even if, however, total initial purchase costs remained about even across the two architecture types the modular design of the "Outside" architecture matches much more readily with ERP and offers greater financial, architectural, and business flexibility as well as investment protection.
Lifecycle Time #4: End-of-Life
End-of-Life refers to the situation that either the hardware or the software has reached the end of its useful function or no longer meets business requirements.
Software end-of-life can occur for a number of reasons:
Hardware end-of-life can occur for a number of reasons:
on the existing hardware
for the split-off entity
These situations, from an infrastructure perspective, are all variants of the situations analyzed previously. They all boil down to the same question, "How granular can infrastructure purchases be in order to meet business requirements?". Clearly, the modular hardware architecture of "Outside-the-Skins" matches more closely with the modular software architecture of ERP and thus allows smaller and finer granularity of expenditures and investment for ERP infrastructure.
In the long term, the most cost-effective tool for a job is the 'right' tool - the tool whose features and capabilities most closely match the requirements of the job to be performed. An "Outside-the-Skins" architecture most closely matches the software architecture and business requirements of ERP.
"Outside-the-Skins" architecture offers superior TCO for ERP
"Outside-the-Skins" architecture offers superior ROI for ERP
"Inside-the-Skins" architectures have some very appealing features and characteristics and as such they deserve review and analysis for environments other than ERP. For ERP, however, there is a fundamental mismatch between software and hardware architectures that drive up TCO and delay ROI. The additional dollars which fund this type of architectural mismatch would otherwise fall directly to the bottom line.
Even more importantly, this mismatch removes a significant amount of business flexibility from your IT infrastructure - flexibility that can allow your business to adapt in a cost-effective way to an ever-faster changing business climate.
Regardless of hardware vendor, ERP's architecture cries out for an "Outside-the-Skins" solution. Of course there will be differences in each vendor's ability and costs to build and support "Outside-the-Skins" solutions for ERP and a thorough competitive analysis should be performed. It is hoped that this paper has provided sufficient insight and guidance for IT and business executives to know the right questions to ask.