Archiving101.com

Since we are entering the holidays I probably won’t be posting as much this coming week as I’ll be spending time with my family enjoying Christmas and all the related activities.   I would like to wish you, my readers, a Merry Christmas to you and your family as well.

While I have to admit that the Stellent Outside In technology isn’t the fastest way, products using it can capture the content of roughly 400 different filetypes.  Products that rely on iFilters (notably Quest Software, Sunbelt and H&S soft) for instance can only capture the content 0f about 60 different filetypes, and to achieve some of these filetypes customers will actually have to purchase extra filters on top of their coslty deployment.  I’m not a fan of iFilters in the archiving industry .. I can see it being valuable in other deployments like SharePoint.

 In case of search and eDiscovery .. this is a huge difference and should be something that customers are aware about.   If you can’t index it .. you can’t find it.

This popped up on one of my RSS feeds last week .. an interesting way to write out an RFI. For the vendors reading this blog …. show that you can be courteous to a customer and eachother

Source: MNET

The Ministry of Foreign Affairs and Trade is seeking expressions of interest from suppliers of email archiving, or Enterprise Vault software, and associated implementation, migration and support services within Wellington.

The ministry has 52 posts in 46 countries with 650 New Zealand staff, 440 of which are in Wellington. A further 190 are based at posts and 20 are on secondment to other organisations or departments. Another 400 people are engaged directly by the posts as local staff.

The ministry’s current IT infrastructure is based on servers running Windows 2003 server software, including exchange 2003 and print services. Desktops are running Windows XP and Office 2003.

There is a Wellington LAN, a wide area network (WAN) linking the posts and a LAN at each post. The Wellington LAN is Ethernet based at 100Mbps. All posts are connected with Wellington via an IP-based WAN connection. With the exception of certain Pacific posts, WAN links are 64kbps per post, with a capacity to burst to 128kbps. The average network roundtrip latency is 500-1000ms, peaking to 2-3 seconds in heavy load. Some Pacific posts are connected by 32kbps links.

The ministry runs the Microsoft Active Directory for user authentication. Hummingbird’s DOCS Open v4 is used for document management within the Wellington office and at each post.

The ministry runs the Microsoft Office Professional suite of products with e-mail connections between all locations, but due to security requirements, there is no desktop access to the internet.

The ministry is developing a Flexible Work Environment, Hummingbird DOCS v6 and Outlook 2007 and would proscribe preferences for MS SQL or dot Net 2. The email system is MS Exchange 2003 and the current archiving product is Legato Email Extender 4.81. This has been in production for two years and the archive has accumulated approximately 1 Terabyte of emails. Only Wellington-based staff have email archived and the replacement system will also be limited to a Wellington implementation.

To answer questions about this RFI, contact Alan Morrison at alan.morrison@mfat.govt.nz. This email address is being protected from spam bots, you need Javascript enabled to view it from Monday, January 14, 2008.

The RFI itself closes 12 noon Friday, 25 January, and responses should be delivered to: The Manager, Management Services Unit, Ministry of Foreign Affairs and Trade. Private Bag 18901, Wellington.

For full details of this RFI, go to the Government Electronic Tenders Service website and search for ICNNZ Reference: 20283.

Having been a Product Manager, I’ve personally been through the grinder of dealing with analysts and how it all works.  The first thing is that some reports are not as valuable as others.  There are vendors who have found and specialized in a niche and are doing extremely well with it.  One of these is the Socha-Gelbmann Annual Electronic Discovery report.  Their 5th report was released last June and is very comprehensive …. 325 pages worth of valuable information. Kudos.  

Now Gartner released their version of their report yesterday and Aaref Hilaly, CEO of Clearwell Systems, lists the top Gartner vendors side by side on his blog.  It is my personal opinion that after seeing the listed vendors, Gartner clearly missed the boat on this one, they for instance mixed service providers and software companies whereas Socha breaks them out separately. The Socha report has an entirely separate list for service providers and therefore is in my opinion much more valuable and representative for this industry.
It has to be noted though thatthis is Gartnersfirst attempt for a report on eDiscovery, but they clearly need to improve on this and look at the benchmark of the industry, the Socha-Gelbmann Annual Electronic Discovery report.   I do appreciate the comments from Socha-Gelbmann they made on the post from Aaref Hilaly. 

With the spur of releases lately there are now 4 vendors that publicly admit they are NOT using MAPI to capture data from Exchange. Notably GFI, Mimosa Systems, MessageOne and H&S Soft (Exchange@PAM) are all using different ways to access Exchange data, while I’ve personally talked with people from other vendors who have admitted that they have working prototypes of their products working with other ways to capture data.

I’ve been around in the messaging industry long enough to understand and acknowledge the fact that MAPI won’t go away over night, even if MAPI hasn’t seen any enhancements since 1998.  After all .. there are more then a million Outlook clients out there connecting to Exchange.  Its an interesting fact though that while MAPI remains the most prevalent way on capture data in the archiving industry, the tide is turning.  I had lunch today with Devin Ganger from 3Sharp who agreed that there has to be a better way to capture the information.  As discussed before .. pretty much what currently is available isn’t really suitable for what we all in this industry want to achieve.  I’m not against MAPI … but for email archiving and compliance there has to be a better way.  The ball is in someones court.

Have a good weekend.

First of all .. I really applaud what Microsoft has been doing with Exchange 2007 .. it brought many features to customers that really have been valuable (I particularly love the logshipping and powershell stuff).   Another feature that was introduced was the transport rules  agent. 

 To quote Microsoft:

Many organizations today are required by law, regulatory requirements, or company policies to apply messaging policies that limit the interaction between recipients and senders, both inside and outside the organization (so called Ethical Walls). In addition to limiting interactions among individuals, departmental groups inside the organization, and entities outside the organization, some organizations are also subject to the following messaging policy requirements:

• Preventing inappropriate content from entering or leaving the organization
• Filtering confidential organization information
• Tracking or archiving messages that are sent to or received from specific individuals
• Redirecting inbound and outbound messages for inspection before delivery
• Applying disclaimers to messages as they pass through the organization

The Transport Rules agent that runs on a Hub Transport server helps you meet each of these requirements. Through the Active Directory directory service, Exchange Server 2007 can apply a consistent messaging policy configuration across the organization. Each Hub Transport server queries Active Directory to retrieve the organization’s current transport rule configuration and then applies that transport rule configuration to e-mail messages that the server encounters. This enables e-mail administrators to set policies across the organization and to implement them on the Hub Transport server as soon as replication occurs.

 I like it that Microsoft at least now puts out the following disclaimer:

Important:
Transport rules can’t prevent people from communicating in other ways, such as networked file shares, newsgroups, or e-mail services that don’t deliver messages to an Exchange 2007 organization.

Now … let me show you another very easy way to bypass an Exchange 2007 Ethical Wall even when there is a MAPI based archiving product being used and complete get away with it undetected (disclaimer .. not telling you to break any laws here).   When you enable journaling you will capture only emails and calendar invites.  Transport rules will prevent you from sending emails through the system so that is out of the question.  Since Journaling doesn’t capture manually created calendar entries in your mailbox all you simply have to do is give read access to your calendar to the other person. Create a calendar entry and simply write the text in there that you’d like to pass on.  Heck .. if you give create/modifying rights you can even have your own calendar chat.  Since most MAPI based archiving products run their ’storage management’ processes out of business hours since it takes a hit on the Exchange Servers, these calendar items chats, if they can actually be scavenged by the archiving product will go completely undetected if you delete the item before the archiving run.  It won’t show up in the audit logs or journal logs.  Again … an area for improvement I have to say.

Interesting … another vendor, MessageOne, (after Mimosa Systems and Exchange@PAM) moves away from Journaling to capture data from Exchange.

http://sev.prnewswire.com/computer-electronics/20071210/LAM009A10122007-1.html

New Capture Mechanism Allows for Targeted Archiving

When litigation occurs, companies must protect historical and ongoing email communications from destruction. Since these capabilities are not provided by Exchange, companies need an archive that enables lawyers to place and remove litigation holds on any user or group at any time.

While litigation typically affects just a small subset of users, most email archiving solutions use Exchange Journaling which requires the capture of every message for all users in a data store. With users spread across multiple servers and data stores, companies are forced to archive messages for everyone to meet litigation requirements for a small number of mailboxes. The result is complex and expensive deployments that can take years to fully complete.

To ease and expedite deployment, MessageOne has introduced the first capture mechanism that does not rely on Exchange Journaling. MessageOne’s on-demand EMS Email Archive integrates natively with Exchange to synchronize messages for any sub-set or combination of users, including those on different servers or storage groups. The quick start EMS Email Archive program allows companies to deploy archiving at once and begin e-Discovery for the most critical users.

“One of the biggest obstacles of archiving is the requirement for companies to complete mammoth, complex and expensive deployment projects,” said Bryan J. Rollins, vice president of product management at MessageOne. “Now, for the first time, enterprise customers can deploy rapidly to any user or group and start archiving immediately.”

Source: http://ca.us.biz.yahoo.com/bw/071210/20071209005029.html?.v=1

FRAMINGHAM, Mass.–(BUSINESS WIRE)–According to IDC’s Worldwide Quarterly Storage Software Tracker, the worldwide storage software market experienced its 16th consecutive quarter of year-over-year growth in the third quarter of 2007 with revenues of $2.8 billion, a 9.8% increase over the same quarter one year ago.

“Overall market growth was driven by strong performance in both the archiving and data protection and recovery markets,” according to Michael Margossian, research analyst, Storage Software at IDC. “Customer demand for storage replication seems to be cooling off, while the need for archiving appears to be picking up.”

The archiving market grew 13.6% year over year, driven by e-discovery, regulatory compliance, and overall storage optimization needs. Data protection and recovery, while down from 2Q07, had strong year-over-year growth of 13.1%. This growth was perpetuated by five out of the top six vendors having double-digit growth over 3Q06.

EMC led the overall market with 25.6% revenue share in the third quarter of 2007. Symantec took the second position with 17.0% revenue share, while IBM finished in the third position with 12.4% revenue share. Network Appliance finished in the fourth position with 10.9% revenue share. Again, CA and HP rounded out the top 5 in a statistical tie, with both having a 4.3% revenue share.

Source: IDC Worldwide Quarterly Storage Software Tracker, December 2007

Notes: Starting in Q2 2007, IBM is reported as the combined entity of IBM and Softek.

Product Management of Archiving vendors are push quite a bit for a release near the end of the year to help push sales in the 4th quarter and as usual .. this season isn’t any different.  One I particular took a look at this morning was GFIs Mail Archiver 5.   Now the product indeed does have a reasonable amount of customers, but I never saw it be any competitive in the more demanding accounts.

I don’t always need to see the software to get an idea of what is good or bad .. the instruction manuals and release notes generally can give you a pretty darn good idea on a product (that is if you know what to look for).  But maybe some of you readers can explain me why on earth you would need to enable IMAP on an Exchange 2007 server to capture journaled messages with GFI ?   Indeed I’m not a fan of MAPI for email archiving .. but I can’t see that IMAP is any better to do this.

 Also .. if you have a release that becomes available over 12 months after Exchange 2007 ships .. why still use ExOLEDB,  a legacy Exchange feature that has been deemphasized in Exchange 2007 and clearly webservices are the replacement for this?

Third .. why do customers who use GFI have to create a new SQL database for each 20GB of data stored in the backend ?  When I had some serious discussions with the SQL development team at Microsoft in the past .. SQL surely could hold more data in a single database then that.

Source: http://www.newswire.ca/en/releases/archive/December2007/06/c8358.html

SCOTTS VALLEY, Calif., Dec. 6 /CNW/ — Today, Seagate Technology (NYSE: STX) announced that it has signed an agreement to acquire MetaLINCS, a technology leader in the fast-growing E-Discovery market. MetaLINCS’s innovative, patent-pending software helps companies respond to litigation and regulatory issues that require them to search large volumes of electronic data for relevant information. MetaLINCS will become part of the Seagate Services Group and support its mission to help business customers protect and manage valuable company information. Financial terms were not disclosed. 

According to Gartner analysts Debra Logan and John Bace in the report, “The Emerging E-Discovery Market,” published on July 18, 2007, “Changes to the Federal Rules of Civil Procedure, along with the ever-increasing reliance on electronic documentation in business, will have wide-ranging effects on the IT profession and IT vendors in 2007 and 2008. IT will be called on to account for elements of their infrastructure and the location of live and backup data as never before.”

“Today’s announcement represents another strategic step for the Seagate Services Group and further reinforces our commitment to providing customers with innovative technology-based services and solutions that advance their businesses,” said Mark Grace, senior vice president and general manager, Seagate Services Group. “The addition of MetaLINCS’s solutions will provide our customers with the tools to respond to litigation and compliance requests and help to reduce enterprise E-Discovery costs.” 

 MetaLINCS’s enterprise-class E-Discovery software automatically analyzes emails, documents and associated metadata, and presents visual analysis of people, conversations, concepts and communication patterns. Founded in 2003, MetaLINCS is a privately held company with more than 50 employees. Ramon Nunez, CEO of MetaLINCS, will join the Seagate Services senior management team and will lead the Seagate Services Group’s E-Discovery business unit. 

 ”We are thrilled to be joining the Seagate Services Group,” said Nunez. “The synergy of our missions and cultures will further strengthen our focus on helping corporations and law firms leverage E-Discovery for strategic decision-making during large-scale litigation, mergers and acquisitions, and regulatory response projects.” 

MetaLINCS is the most recent acquisition for the Seagate Services Group.  In 2007, Seagate purchased EVault, an award winning provider of online backup and archive solutions for small and mid sized enterprises, and in 2005, Seagate purchased Action Front (now Seagate Recovery Services), a leading provider of data recovery and data migration services. These acquisitions provide Seagate with growing opportunities in the data protection and management solutions market and are highly scalable with Seagate’s technology portfolio and market expertise.

With the addition of MetaLINCS’s E-Discovery platform, Seagate Services is able to offer corporations, law firms, and litigation service partners a technology leading analytics engine along with one stop sourcing for archive, recovery and collection, review tools and services inclusive of EVault’s Insight E-Discovery services.     Following the MetaLINCS acquisition, the Seagate Services Group will provide the following primary solutions: 

- E-Discovery solutions targeted at addressing a growing market need to retrieve and analyze large volumes of data in support of litigation and compliance requirements
- Complete business continuity, compliance and disaster recovery services with high availability, secure backup and recovery of business-critical data
- A wide range of data recovery, data migration and data accessibility solutions to meet the needs of enterprise, small/medium businesses and
consumers