Phase 1 - establish criteria for evaluating exhibited vendors, the Evaluation Matrix
* Extra special thanks to Alex Krylov for compiling these notes *

Purpose
NYBI's mission is to present an unbiased evaluation of the BI vendor space through presentations, discussions, and targeted publications. By providing an open forum for a diverse group of professionals we strive to understand trends, innovation, implementation and application of Business Intelligence. Above all else, we seek to engage with and enrich the BI Community.

Key Ideas
• A community-driven understanding of BI space vis-a-vis hands-on perspective - "Our biased view".
• "What can we as practitioners say about the BI tools?" Looking for "intelligent feedback to each vendor & community base."
• Evaluation not all-inclusive
o Missing Oracle, Microsoft, SAS [vertical monitoring / modeling (?)], etc
o MS Sharepoint not necessarily relevant, but does offer BI features
• Visualization as current focus; analytics later

Business User vs Implementor
• Typical users split between wanting something Excel-focused and a server-based solution b/c
o Excel is the standard for most analysts
- Many analysts prefer to dump data into Excel and manipulate data themselves (pivots, etc)
- Many analysts are comfortable w/o a data warehouse (use copy of transactional system or aggregate data from transactions themselves)
- Managers / Execs tend to hire "Excel Monkeys" to generating custom reports
- Excel users spend most of their time digesting large chunks of data in Excel, documenting how they do this, and then manually creating reports
- "People will use what they are most comfortable with - Excel"
o Excel provides flexibility, but limits standardization as each analyst does things differently
• "When do I speak with IT? before or after?"
o IT / DBA has burden of supporting custom report requests
- Reports still get exported to CSV or XLS
- Managers dissatisfied with turnaround time
- IT dissatisfied with responsibility beyond implementation and general support (intervention)
- IT interested in level of prep / plumbing behind the scenes

The Value Proposition
• How do BI vendors we've seen so far approach / deal with the Excel monopoly?
o Do they work with it (use as jumpboard) or try to minimize it?
o Do they strive to surpass its core capabilities or enrich it?
o Do they offering new tools out of the box or a Googlified XLS?
o Do they focus on analytics or cool reporting widgets? (Prep vs Presentation)
o When would user use the BI product? (workflow & appropriateness issue)
o How does the BI vendor treat Security?
- Everything locked down?Controlled procedures?
- IT tends to build out first, think of controls second

Brainstormed Concepts and Dimensions organized into Buckets

*Madhavan's ontology to be used to define and standardized terms (NYBI Dictionary)

I've converted Forrester's BI Architectural Stack to excel format. It is available to this Meetup here: http://files.meetup.c...

What Forrester is saying about:

TIBCO Spotfire
Actuate, MicroStrategy, and TIBCO Spotfire go after best-of-breed components.
TIBCO, a recent entrant into the BI market with the acquisition of Spotfire, leverages its long history of success and expertise in middleware for integrating visual information analysis with process orchestration and business rules.

QlikTech and TIBCO Spotfire invented “BI without borders” — lightning-fast access to large in-memory data models, where recalculations, re-aggregations and what-if analysis can be performed with few limitations. In-memory architectures do not require prebuilt data models, so any element can be re-examined as a fact or a dimension on the fly.

TIBCO Spotfire differentiates with visualization and integration with process and rules. While many BI vendors talk about enterprise optimization or enterprise decision management that can only be enabled through seamless convergence of BI, business process management, and business rules engines, the TIBCO Spotfire team is actually implementing such solutions today.14 Its Operations Analytics bundle can display data and process metrics and let an end user correct rules and process flows if necessary. Users can even see the updated results in real time. This is also partially possible by Spotfire’s powerful in-memory analytical engine, which, similar to QlikTech, can update results in memory with lightning speed. Furthermore, Spotfire continues to build on its original strength and differentiation: advanced data visualization functionality that helps analysts identify data characteristics and trends based simply on visual pattern recognition.15 TIBCO is very serious about BI, and we expect more acquisitions or more internally developed products to round out its BI portfolio in the near future. Today, however, it
continues to rely heavily on partners to deliver complete end-to-end BI solutions, which always need to include data quality and other products from partners.

XCelsius
There’s indeed a best-in-class tool for every use case: Crystal for production reporting,
WebIntelligence for ad hoc querying, Voyager for OLAP, Polestar for guided BI search, Xcelsius for interactive dashboards, and several more.

QlikTech
Panorama Software and QlikTech still lack breadth, but they shine in specific use cases. Even though Panorama Software and QlikTech lack the breadth of options and features needed to be all-encompassing enterprise BI solutions, they offer highly differentiated tools for certain specific BI use cases.

QlikTech and TIBCO Spotfire invented “BI without borders” — lightning-fast access to large in-memory data models, where recalculations, re-aggregations and what-if analysis can be performed with few limitations. In-memory architectures do not require prebuilt data models, so any element can be re-examined as a fact or a dimension on the fly.

This is also partially possible by Spotfire’s powerful in-memory analytical engine, which,
similar to QlikTech, can update results in memory with lightning speed.

It’s not the strategy of smaller BI vendors like Panorama Software and QlikTech to address BI components other than reporting and analytics. Until they get acquired (which is almost inevitable) these vendors will continue to rely heavily on partners to deliver complete end-to-end BI solutions with data integration, data quality, and other BI stack component providers. However, these smaller BI vendors also continue to demonstrate that there are still plenty of opportunities to innovate in the vibrant BI market segment, which is quite far from maturing and commoditizing.

QlikTech popularizes “BI without borders.” No matter what BI architecture or solution
one chooses, a data modeling step is still required before reporting and analysis can be
performed. In-memory data models from QlikTech (and TIBCO Spotfire) do not require that precondition, since all calculations and aggregations can be done at RAM speeds and therefore require little, if any, design and preparatory work. There are times when one needs to treat a continuously variable numerical value (like sales price) not as fact, but as a dimension. While conventional OLAP tools make this very difficult, in-memory models do not require a distinction between facts and dimensions — any element can be instantaneously used in either capacity. QlikTech spent years optimizing memory utilization and now can take full advantage of 64-bit architectures with in-memory models as big as a terabyte (as opposed to the 4 gigabyte limitation of 32-bit computers). While QlikTech does not have the breadth of functionality necessary to be the only enterprise BI solution, its lightning-fast response times for instant exploratory analysis and what-if scenarios, including on-the-fly re-aggregations, can be very attractive to power users and analysts.

MicroStrategy
MicroStrategy still outperforms the competition with a truly unified platform and the top ROLAP engine, which is very competitive — especially in environments with multiterabyte data warehouses or poorly architected or optimized reporting databases.

MicroStrategy still shines as a truly unified platform with the top ROLAP engine.
MicroStrategy, one of the few BI vendors that have not acquired any technologies, developed 100% of its products in-house and as a result offers a truly unified platform. What it means is something few other BI vendors can offer: You can define any object (a prompt, a filter, etc.) only once and reuse it in multiple reports and dashboards. When you need to change that object, you change it only in one place. Also, unlike many other BI vendors that rely mainly on the power of the underlying DBMS, MicroStrategy offers its own very powerful ROLAP engine, which can virtually optimize even poorly architected data models. While many other BI vendors partner with third-party providers for higher-end analytics such as statistical analysis and predictive modeling, MicroStrategy has such functionality built in and fully integrated. The price: It is not MicroStrategy’s strategy to address BI components other than enterprise reporting and analytics, and the company will continue to rely heavily on partners to deliver complete end-to-end BI solutions. As a result, MicroStrategy buyers often
need to assemble components from other vendors.

Url: http://www.sas.com/ne...

EXECUTIVE SUMMARY
In Forrester’s 151-criteria evaluation of enterprise business intelligence (BI) platform vendors, we found that IBM Cognos and SAP Business Objects maintain their leadership positions, while Oracle and SAS Institute move into leadership positions in enterprise BI thanks to the richness of their functionality, ability to scale, and the completeness of their corporate and product vision and strategy. Actuate, Information Builders, Microsoft, MicroStrategy, SAP, and a new entrant, TIBCO Spotfire, came out as Strong Performers following very closely on the heels of the Leaders, offering very respectable alternatives and a multitude of choices for information and knowledge management (I&KM) professionals. New entrants to this Forrester Wave™ evaluation of enterprise BI platforms (though not
new to the market) are Panorama Software and QlikTech, who, while lacking the breadth of features to qualify them as single, large-enterprise BI solutions, are reputable Contenders. In some very specific BI use cases, they even outperform the Leaders.

Research Notes:
Forrester conducted lab-based evaluations in March 2008 and interviewed 12 vendors and 24 user companies, including Actuate, IBM Cognos, Information Builders, Microsoft, MicroStrategy, Oracle, Panorama Software, QlikTech, SAP, SAS Institute, and TIBCO Spotfire.

Role of IT
“I do not want IT solutions to be just about productivity and efficiency gains. If we are not productive and efficient at this point, we are in the wrong business. I need my IT colleagues to be real business partners and bring solutions that will enable our company to win new business, convert more prospects to customers, increase our market share, and get into the new markets. I need IT to be a true business partner, not a cost center.”

-> product are commoditized, biz competing on analytics

Quotes
"The BI market, while consolidating, is far from being mature."

Dominant BI Product Functionalities
• Production/operational reporting for pixel-perfect mass report distribution. Used by Professional Developers
• Ad hoc query tools provide a quick answer to a business question.
• OLAP tools, when business questions are more about “whys” than “whats”.
• Dashboards as an interactive, visual UI — not a reporting or analytical tool by itself.
• BAM to report on real-time data and process information streams.
• Predictive modeling answers questions about what’s likely to happen next.
• BI workspaces enable true end user self-service. -
Forrester defines a BI workspace as a data exploration environment where
a power user can analyze production, clean data with near complete freedom to modify data models, enrich data sets, and run the analysis whenever necessary, without much dependency on IT and production environment restrictions.
• Guided BI search tools support free form ad hoc queries and analysis.
This
effectively solves one of the oldest dilemmas in BI: having to know exactly which questions to
ask to get a meaningful answer

Evaluation Criteria
• Current Offering
§ Architecture
§ Development environment
§ Functional capabilities
§ Operational capabilities
• Strategy
• Market presence

Vendor Selection
• Does the vendor’s product have at least three out of the four major functional BI components?
• Does the vendor have the ability to query databases using SQL and MDX?
• Does the vendor offer a self-contained, complete, fully functioning BI environment?
• Does the vendor have sufficient market presence and interest from Forrester clients?
• Does the vendor have at least $40 million in BI revenues?

Vendors
Leaders: SAP BO and IBM Cognos in lead.

Differences in philosophy:
• SAP BO - best of breed for every use case -> acquisition
• IBM Cognos - unified integrated platform

Runner up:
• SAS
remains the best game in town for fully integrated high-end analytics from a single vendor.
• Oracle
Acquisitions of Siebel and Hyperion technologies

Information Builders:
Microsoft moving with dizzying speed, competing with SAP.

Actuate, MicroStrategy, and TIBCO Spotfire go after best-of-breed components.
...when all one is looking for are best-in-class reporting and analytical tools, and integration with the rest of the BI stack components is less of an issue…

• Actuate - more and more attractive to enterprises seeking to leverage open source alternatives
• MicroStrategy still outperforms the competition with a truly unified platform and the top ROLAP engine, which is very competitive — especially in environments with multiterabyte data warehouses or poorly architected or optimized reporting databases.
• TIBCO, a recent entrant into the BI market with the acquisition of Spotfire, leverages its long history of success and expertise in middleware for integrating visual information analysis with process orchestration and business rules.
• QlikTech - differentiated product for certain specific BI use cases.

QlikTech and TIBCO Spotfire invented “BI without borders” — lightning-fast access to large in-memory data models, where recalculations, re-aggregations and what-if analysis can be performed with few limitations. In-memory architectures do not require prebuilt data models, so any element can be
re-examined as a fact or a dimension on the fly.

Not all solutions are created equal, however, and the lines of demarcation for shortlisting vendors are still pretty apparent: rich functionality versus integrated platform; suite versus best-of-breed; stack-integrated versus stack-independent tool set; more expensive solutions offering near 100% functionality versus less expensive ones following the 80/20 rule; and many others.

Vendor Profiles
• Business Objects, an SAP company, provides some of the best-in-class tools for each use case.
• IBM Cognos provides one of the most modern and scalable BI environments.
• SAS is the best one-stop BI shop that includes high-end analytics from a single vendor. Hard to beat in text and statistical analytics.
• Oracle brings the best of its rich BI portfolio into Enterprise Edition Plus.
• SAP NetWeaver BI is still best in SAP shops, but Business Objects will change all that.
• Information Builders continues to innovate, offering the most extensive data access adapters.
• MicroStrategy still shines as a truly unified platform with the top ROLAP engine.
• Microsoft is becoming enterprise-grade with dizzying speed.
• Actuate still differentiates on a sophisticated production report development environment.
• TIBCO Spotfire differentiates with visualization and integration with process and rules.
• Panorama Software is a great option when all you need is MDX.
• QlikTech popularizes “BI without borders.”