EDRM and SharePoint: Designing and Building a Compliant Platform

The amount of information generated by organizations is growing at a phenomenal rate. With factors such as increased legislation and penalties, many organizations want to leverage their information assets, both physical and digital, to their full advantage by increasing speed of access, improving classification schemes, and assuring information quality and trustworthiness. To do so, organizations are adopting toolsets such as Electronic Document and Records Management (EDRM) applications, Microsoft SharePoint, and other document management tools for creating a central trusted repository, which is a key goal of records managers.

There are many roadblocks to attaining that goal, such as executive sponsorship and enforcement, organizational alignment, confusion over toolsets, lack of a consistent and user-relevant classification scheme, poor usability, and low user adoption. Most organizations have islands of EDRM systems in various states of deployment that are generally not delivering the value they promised. That's why having a good planning and deployment strategy in place is crucial. 

 

Planning your Records Management Solution

Planning a records management solution is a daunting task. To be successful, you need a multidiscipline team because designing the system requires multiple skills and diverse knowledge (e.g., communications, governance, information architecture, records management, data migration, security, EDRM system architecture). A good place to start the planning process is to read ISO 15489 and the DIRKs methodology. (See the Related Resources Box.) Here are some key points that you need to consider when planning your EDRM rollout:

• Governance—You need an executive sponsor to drive your records management program across the organizations.
• Team—You need to work with a multidiscipline team as discussed earlier.
• Plan—You must have a plan that will deliver the expected value for the organization.
• User adoption program—You must create a program that encourages and measures user uptake, provides education and mentoring, and enforces and enforces best practices (performance reviews).
• Data policy—You must define, communicate, and enforce the data policy for records. People must understand their roles and responsibilities.
• Inventory of data repositories—You need to conduct an inventory of what you have, the state it's in (age, usefulness), and location. The inventory might include file shares, public folders, microfiche, third-party records storage warehouse, and departmental EDRM systems to name a few.

 

Records Center 2007 Overview

Although Microsoft has released SharePoint 2010, there is merit in covering SharePoint 2007 technologies because many organizations won't upgrade for another year or so. SharePoint 2007 has a Site Template called Records Center. It has functionality such as Holds, which lets you place holds on retention schedules, and Auditing, which logs events and operations against the record. It also contains some specialized libraries and content type routing configuration that allows you to route records by content type. 

For example, you could create a content type called Contracts and a library for the content type. You would then create a routing rule in Records Center that would route the content type to the Contracts library. You could then apply Information Management policy (i.e., how to handle records) to the library. The final step in the process is to configure the farm to be aware of the Records Center by adding the link to the Records Center Web Service (e.g., http://server_name/_vti_bin/officialfile.asmx). To do so, select External Service Connections and enter the URL to your records center web service in the Connect to a Records Repository field. Upon completion, the option to route records within document libraries will now appear in the item's edit/context menu as Figure 1 shows. Note that any work you do should be done in a sandbox environment, which is usually a virtual machine (VM) running on your laptop.

 

Create a Records Center

This section outlines the basic process for creating a Records Center in Microsoft Office SharePoint Server (MOSS) 2007. For more detailed information, see the Microsoft TechNet article "Create a site collection (Office SharePoint Server)". To create a Records Center, follow these steps:

1. Login as Administrator and open Central Administration. Select Application Management.
2. Enter a name for the application, SQL credentials, and authentication method that suits your environment). Using Quotas is also a best practice, but isn't a requirement for a sandbox environment. Note that isolating Records Center with its own site and application will help simplify operations.
3. Under Site Collection, select create a Site Collection. Enter a name, description, and URL, then select the Records Center template. (Chose a URL and port number that fits with your environment. Also, associate your Site Collection with the application you created in the prior step).
4. Select Application Management and click External Service Connections. Enter the URL to your Records Center Web Service in the Connect to a Records Repository field (e.g., http://server_name/_vti_bin/officialfile.asmx). Note that in SharePoint 2007 you can specify only one connection per farm.
5. Open your Records Center site in the browser and click Site Settings. Create a content type called Contracts, and accept the default settings. Click Site Settings, select Create, and create a document library called Contracts. Complete the following tasks:
6. Configure the Contracts document library to accept multiple content types. Configure the Contracts document library to use the Contracts content type you created in the previous step.
7. Create a routing rule called Contracts that directs the Contracts content type to the Contracts document library by specifying the name of the library as the destination.

You now have a functional Records Center site. If desired, you can add the Records Site to your Portals navigation by editing the Portals navigation menu and adding a tab for the new Records Center site.

 

Records Center 2010 Overview

In SharePoint 2010, the Records Center has a slick new UI and some additional features that improve usability and management. For those who evaluated SharePoint 2007 and held off, SharePoint 2010 offers compelling new features and enhancements. Here is an overview of some of the new functionality.

      In-place records management. A new industry trend is the idea of in-place records management. Specifically, instead of a central repository of documents that requires a routing service or manual movement of files, records are managed where they are. The documents stay in the current location and are classified as business records. This approach avoids the laborious migrations associated with EDRM projects.

      Multistage retention. Retention policies can have multiple stages, allowing you to specify the entire document lifecycle as one policy (e.g., review Contracts every year, and delete after seven years).

      Folders. Another interesting note is the new role of folders in SharePoint 2010. In SharePoint 2007, folders were the recommended option for grouping files by security access or to be avoided. With SharePoint 2010, folders can act as true parents to any child objects below it. Now, metadata is set at the folder level so that the child objects can inherit that information.

      Document sets. Document sets enable users to collaborate on related documents without having to create a new document library or site. Their purpose is to help organize documents and enable collaboration on documents tagged with similar metadata.

      Persistent document IDs. The document ID provides absolute reference to objects, regardless of filename changes or document moves.

      Content organizer. The routing rules from SharePoint 2007 have been replaced by the content organizer, a new SharePoint feature available in all document libraries. The content organizer routes documents to the right folder based on content types and any other metadata that you require.

      Compliance details. This is a new feature that adds a Compliance Details option to the context menu. It allows users to check out the relevant settings that have been applied to a specific business record. The feature also enables administrators to make sure that specific documents are inheriting the right policies and retention settings.

      Submit a record. This is a basic feature that adds some simple usability to the site.

For more information, read the article “8 Reasons SharePoint 2010 Looks Like a True ECM System”.

 

Creating a Records Center in SharePoint 2010

1. The following steps provide the basic outline for creating a Records Center in SharePoint 2010. To find more in depth information, see the TechNet article "Create a site collection (Office SharePoint Server)".
2. Log in as Administrator and open Central Administration. Select Application Management.
3. Open the Create Site Collection page. In the Web Application section, if the web application in which you want to create the site collection isn't selected, on the Web Application menu click Change Web Application, then click the web application in which you want to create the site collection. TechNet has a detailed article about this process, "Create a site collection (SharePoint Server 2010)".
4. As with SharePoint 2007, create a web application for Records Center. Chose a name, SQL credentials, and an authentication method that suits your environment.
5. On the Create Site Collection page, in the Web Application section, if the web application in which you want to create the site collection isn't selected, on the Web Application menu, click Change Web Application, and click the web application in which you want to create the site collection.
6. In the Title and Description section, type the title and description for the site collection. In the Web Site Address section, select the path to use for your URL (e.g., a wildcard inclusion path such as /sites/, or the root directory). In the Template Selection section Select Records Center from the list. Click OK.

Note that SharePoint 2010 enables administrators to create multiple connections to Record Centers per farm, and to specify more detailed parameters about the behavior of those connections. Another enhancement is the ability to move an item to the Records Center, or to move an item, but leave a link in the original location.

Next you need to create content organization rules. You add rules by following these steps:

1. Select Site Settings, Site Administration, Content Organizer Rules. Then select New to open the New Rule page.
2. Create a rule called Contacts. Page down and select the Target Location. Browse to a Library in your site. Make sure that whatever content type you choose is activated in the target library, or you'll receive an error that the destination doesn't have the content enabled.

It's important to note that the source and destination libraries (i.e., document libraries configured for inplace records mgmt) will require the same content types to used. For example, if you have a Contracts Form content type in your source library, you will also need one in your destination library.

 

In Place Records Management

To take advantage of In Place Records management, you must first activate the feature for the site collection, as Figure 2 shows. If you have a site collection configured in your sandbox environment, go to Site Actions and activate the feature. Otherwise, create a site collection (e.g., Team Collaboration), then activate it.

 

If you want items within the libraries to be automatically flagged as records (e.g., a Contracts library), choose the library in your site that you want to enable to support in place records management, and click Library Settings. Enable Automatic Declaration. Now, all documents added to the library are automatically flagged as records. Note that after you activate the feature and configure the library, a new button, Declare Records, appears in the Libraries ribbon. You can read more in the TechNet articles "Records management overview (SharePoint Server 2010)" and "Designing for in-place records management".

 

Third-Party EDRM

Butler Group released a review of the top records management vendors and their products in late 2008. EMC Documentum, IBM FileNet, and OpenText were the notable top players. HP TRIM is classified as Consider, and SharePoint 2007 is classified as Explore.

An important pattern to note is the rise of SharePoint. It’s disrupting the EDRM market as well as some others such as Web 2.0. With its high cohesion with Office, its ease of use (generally a high rate of adoption), and its low cost from a licensing perspective, SharePoint is a good option for more and more companies. According to Butler Group, “SharePoint is designed to help organizations manage content while enabling information workers to communicate and collaborate electronically.” EDRM vendors have made attempts to integrate with SharePoint using simple web parts that expose the File Plan and provide varying levels of search integration. The web parts typically include:

Document library. This web part enables Site Administrators to expose the EDRM File Plan as a document library. You need to specify the area (Container/Class) of the File Plan to expose it. This web part can make the provisioning process more difficult because of the skills required to configure it when provisioning sites; therefore, adopt with caution.

The Edit/Context Menu will include the “Move Document Feature” option. This web part adds a feature to the Edit/Context Menu that enables users to select a document to be moved to the EDRM File Plan. It generally exposes an ASPX page that requests record type and required metadata before sending to the EDRM.

Search protocol handler. This web part is installed on the Index Server and enables the server to crawl third-party applications such as an EDRM. For SharePoint to display combined search results from a content source, it must understand how to access, crawl, and return results. This is the job of the handler. I've found handlers to perform slowly and sometimes hang the indexer. Adopt with caution.

Federated search web part. If vendors don't have the time or expertise to develop a handler, the federated search web part is available. SharePoint search passes the web part, receives the search criteria from SharePoint search, processes the query, and displays it as a separate results set. I've found these web parts easy to install, and they perform well.

I’ve worked with a few of the EDRM's products web part integration and was not impressed. They don't provide true integration, but instead are more or less a simple gateway to the EDRM via web parts. They don't provide the ability to take advantage of features such as alerts, work flows, and views. Indexing performance is generally slow and doesn’t scale well. There are often stability concerns and provisioning is problematic. Given its almost viral growth rate, SharePoint will continue to disrupt the EDRM market as it did the Portal market.

 

To the Future

To understand how the way people communicate has changed over the past century, and especially in the past 20 years, think about the communications media that past generations used: letters, phone, and fax and of course face-to-face communication. With these forms of communications came a simpler slow-paced work environment than we experience today.

Today, we use a variety of devices and applications such as smartphones, email, IM, cameras, Facebook, Linked In, VOIP, web conferencing, Twitter, and corporate systems such as email and voicemail.

EDRM is no longer the simple capture of physical and electronic assets such as mail and electronic documents. The scope of EDRM spans many applications and communication media and is far reaching organizationally. Some organizations realize this and have locked down corporate access to Internet applications. But what are the employees communicating from home? Are they leaking company secrets? Are there inappropriate communications about the organization or its people? Are recruiters chasing your staff on recruitment and networking sites?

Tools such as SharePoint technologies and Web 2.0 have changed the EDRM market. SharePoint 2010 is applying pressure to EDRM systems to step up their game regarding in place records management. Records managers and IT managers are thinking of ways to capture communication in applications such as Facebook and Twitter. The price point and usability of SharePoint is pushing EDRM tools out and changing the way companies think of and manage information assets. I predict that standard EDRMs will become a backend to software such as SharePoint. EDRM vendors will improve their integration by creating deep linkages to applications such as SharePoint and Web 2.0 applications that are seamless to the average user. Users won’t even know their documents and communications are being captured as records.