Lotus Notes to SharePoint Blog

Blog about Dell's Notes Migrator to SharePoint tool and other things related to Lotus Notes migration projects

Category Archives: Version 6.2

Important notice regarding SharePoint 2013 support in Notes Migrator for SharePoint

When we shipped Notes Migrator for SharePoint 6.2 last month, this included support for SharePoint 2013 but we realized that had one very unfortunate limitation: 

The Import Service did not support SharePoint sites that used Claims Based Authentication.  This was actually not a new limitation, and it impacted a handful of SharePoint 2010 customers, but it suddenly became very important in SharePoint 2013.  The reason is that all SharePoint 2013 sites use claims by default.  So this effectively meant that most SharePoint 2013 users would be forced to migrate via the native SharePoint web services.  Not a terrible option, but definitely not as fast.  (Migrating via the Import Service can be 3 to 5 times as fast, especially if you have large attachments and use the Share Files Folder feature.) 

Happily, we have now resolved the limitation.  Starting with hotfix build 6.2.0.1060, The Quest Import Service will work as expected even with the new SharePoint 2013 defaults.  This fix will appear on our web site in our next point release (6.2.1), but many customers will not want to wait for that.  Therefore I would encourage anyone who wants to use the Import Service on SharePoint 2013 to proactively contact Quest support and request build 6.2.0.1060 (or any hotfix build after that).   

Filter By Rich Text Content

In SharePoint, document libraries contain files. Every “document” is a binary file that you can download to your hard disk, etc. Notes document libraries are more flexible however. Some documents contain just one file attachment, while others may contain lots of rich text (and possibly multiple file attachments) in a rich text “Body” field. In fact, it is common to see mixed usage patterns in one document library, making it difficult to figure out the best migration target.

In the past, the best approach has been to migrate document libraries to a SharePoint list instead of a document library. SharePoint Lists mirror the flexibility of Notes documents well and allow you to capture your rich text (if any) in a “Body” field and migrate zero, one, or many attachments to the list’s attachment area. Unfortunately, you lose all the advantages of SharePoint document libraries with this approach. In the cases where you know that most documents are really just single file attachments, you would probably prefer to migrate those directly to a document library.

What many migration teams want to do is apply the following policy for document libraries:

  • For documents that contain just one attachment (and no other rich text), migrate the attachment directly to the SharePoint document library with all the appropriate security and metadata.
  • For documents that contain Notes rich text, generate a Word or PDF document and place it in the same SharePoint document library with all the appropriate security and metadata.
  • For documents containing neither attachments nor rich text, either skip the document or create a stub entry in the target library.

In order to implement this policy, Notes Migrator for SharePoint 6.2 now includes a new record filtering option for Notes and Domino.Doc data source definitions.

image

On the record selection tab, check the “Select documents based on Rich Text Content” checkbox. This will enable a Details button where you can specify further details. First specify one or more rich text items you would like to inspect. Second, specify the criteria you would like to use for filtering documents:

  • Whitespace only
  • One attachment only
  • Multiple attachments or other rich text

This new record selection option allow you to create multiple migration jobs for each document library, each one implementing one of the rules in the above policy. Remember that the Notes Migrator for SharePoint migration console makes it easy to sequence multiple migration jobs for one database, and to automate these jobs for many databases of the same type.

Also note that we are planning a similar feature for extracting QuickPlace and QuickR folders, but this is not available in the current build.

Enhanced support for Lookup Fields in Notes Migrator for SharePoint 6.2

In version 6.0 we added support for migrating content to SharePoint lookup columns. In version 6.1 we added provisioning of new lookup columns. Now here are two new advanced choices for using Lookup columns to help you design more powerful migration jobs.

Add Missing Choices

Normally when you migrate content to a SharePoint List that contains a lookup column, the Notes Migrator for SharePoint code tries to find the data value and correlate it a record in the lookup list. For example, you may be mapping a VendorCode item from your Notes data source to the “Vendor Code” column in your SharePoint target list, which is lookup column that connects to an indexed Vendor Code column in a list containing Vendor records. If you set the new “Add Missing Choices” property in your column definition to true, we will add a new record to the Vendors lookup list if one does not already exist. (This is now very similar to how “Add Missing Choices” works for Choice columns in our tool.)

image

Note that in order for this to work, the Lookup list should have reasonable default values for all the other columns (especially the ones that are required columns).

Override Lookup Column

A mapping such as the one described above (mapping a VendorCode item from your Notes data source to a Vendor Code column in your lookup list) is fine for simple cases. But sometimes the situation is more complicated than that, and you need to use some other value from the Notes data source to correlate the data.

Suppose for example that the Notes application used VendorTaxID to link things together but the new SharePoint template is designed to use Vendor Codes. With version 6.2 you can now specify an “Override Lookup Column” property to control how Notes Migrator for SharePoint locates the record in the target list to link to. In the following screen shot, note how the Tax ID column in the lookup list overrides the Vendor Code column that is configured in SharePoint. Now you can map VendorTaxId values from Notes and get Vendor Codes in your SharePoint lookup column in the end.

image

Another great use for this new feature is the case where the Notes application used response documents to link two types of records. For example, it is common to use response documents to link user Comments to a patent document, such as a team’s Action Items (shown below).

image

In order to correlate the Comment documents to the parent documents, we need to use the internal Notes universal IDs (a.k.a. UNIDs). As luck would have it, Notes Migrator for SharePoint always stores the UNID of each migrated Notes document in a hidden “NotesUNID” column in SharePoint. So you can use that column as your Override Lookup Column and simply map the parent IDs to the lookup column:

image

Note: this technique works perfectly with the SharePoint Blog site template. The Comments list has a lookup column that references the Posts list. Now you can correlate Comments with Posts using internal Notes UNIDs, which is a lot more reliable than using Titles (which may contain duplicates).

Direct Folder / Document Set Migration

In previous versions of Notes Migrator for SharePoint, users could map document metadata (for example the Category property of a Notes application or the {BinderName} property for Domino.Doc documents) that would cause folders to be created in SharePoint. Folders would be created as needed as documents were being migrated. This worked for most cases, but there were limitations that customers would occasionally ask about:

  • Since we only migrate folders as a “side effect” of migrating documents, there was no way to migrate empty folders.
  • Similarly, there was no way to create the folders ahead of time (before migrating the documents)
  • There was no way to set permissions, created/modified metadata, or additional data columns on the newly created folders

Now in Notes Migrator for SharePoint 6.2 we offer a way to do direct folder migration. This is really two separate features that work together…

Migrating records to folders

image  image

On the Advanced tab of your Target Data Definition, you can now indicate that you want to migrate to a folder in your target list or library.  In this mode of operation, every record you extract from the data source will result in a folder being created, instead of a document!  The only additional requirement is that you map at least one item to a target column of type Folder (which controls the new folder names). Many of the usual document migration features will now apply to folders including:

  • Mapping of permissions (using the “Map Reader/Author fields” checkbox on your Advanced tab)
  • Mapping created/modified metadata to folders (using the “Preserve Created/Modified” checkboxes on the Map Data tab)
  • Mapping additional data items to folders (requires creating a new Folder content type). 

Note that many job features that would apply to document migration will not apply to folder migrations. For example, document generation and duplicate document handling options would be disallowed in in this context.

Extracting information from Domino.Doc Binders

image image

One of the things that customers clearly want to do with this new feature is to migrate all the information from their Domino.doc Binders to SharePoint folders. To support this, we have added a new option to do exactly that in Domino.Doc Source Data Definitions. Simply check the Binders radio button on the Document Selection job, and now you are extracting Binders instead of Documents. Each row in the Preview represents a Binder in the current file cabinet and we have included additional columns for all of the standard Binder metadata available in Domino.Doc. Of course you can add additional columns to this query as well.

So putting these features together, you would typically map the {Title} property of your data source to a Folder column in your target. Simply checking “Map Reader/Author fields”, “Preserve Created/Modified identities”, and “Preserve Created/Modified dates” should bring over most of the other metadata but you can certainly add additional mappings if desired.

Note that this feature will only write new SharePoint folders; it will not update existing ones with the same name. So a best practice is to run the Binder migration job first (to create the folders with all the properties intact) and then run you normal document migration job.

Also note that we are planning a similar feature for extracting QuickPlace and QuickR folders, but this is not available in the current release.

Migrating to Document Sets

Similar to migrating to folders, Notes Migrator for SharePoint 6.2 also give you the ability to migrate directly to document sets. The situation here is very similar to what was described above. The tool already allowed creation of document sets as files within those document sets were being migrated. This is a powerful and popular feature, but suffered some of the same limitations.

  • Since we only create document sets as a “side effect” of migrating documents, there was no way to create empty document sets.
  • Similarly, there was no way to create the document sets ahead of time (before migrating the documents)
  • There was no way to set permissions or created/modified metadata on the newly created document sets separately from the documents.

The solution is similar to the folder solution described above. On the Advanced tab of your Target Data Definition, you can now indicate that you want to migrate to a document set in your target list or library.  In this mode of operation, every record you extract from the data source will result in a new empty document set being created, instead of a document!  The only additional requirement is that you add a target column of type DocumentSet and map a value to the DocumentSet.Name property. All of the other features of document set migration (described here) still apply. The difference is that every record you select gets mapped to a document set instead of a file within a document set.

Automatically ZIP file attachments

Many customers have requested the ability to compress Notes file attachments while migrating them to SharePoint. There are a number of good reasons for wanting to do this:

  • Save disk space on SharePoint server
  • Get around SharePoint file restrictions (i.e., blocked file extensions and/or size limits)
  • Improve the bandwidth sending data to remote SharePoint servers
  • Eliminate problems (hangs and memory leaks) when embedding certain types of file attachments inside Word documents

Notes Migrator for SharePoint 6.2 now allows you to achieve this in Notes and Domino.Doc migration jobs via a new property on Attachment columns in your Source Data Definition. The “Compress” property may be set to “None” (the default) or to “Zip”.

clip_image002

You can also configure a set of global exceptions to this rule on the Notes tab of the tool’s Options dialog. The “Compression Exclusions” option allows you to specify any file extensions that should never be zipped. This would typically include media files that are already well-compressed and would not benefit from zipping.

clip_image004

When the Compression property is set to “Zip”, all extracted attachments will be compressed and placed inside a ZIP file when migrated. As shown below, any icons or text links to the attachments will appear the same as before, but when the user clicks on them they will open up a ZIP file instead of the “raw” file.

clip_image006

Additional Notes

Zip files will always be excluded from further zipping, even if they are not specified in the “Compression Exclusions” list.

If you wish to use this feature to migrate files that were previously blocked by SharePoint (for example .EXE files), be sure to remove these extensions from your Blocked Files list on the SharePoint tab of the tool’s Options dialog.

When using this feature with Domino.Doc sources, add a custom Attachment column as described above. Map this custom column (instead of the predefined {Attachments} column) when mapping attachments to a target column.

clip_image008

This feature is not currently available to QuickR or QuickPlace.

Introducing Notes Migrator for SharePoint 6.2

This release will be downloadable from the web site in a couple days. 

The biggest feature by far is full SharePoint 2013 support.  Everything you could do in SharePoint 2010 you can now do in SharePoint 2013 and the new version of Office 365.  Its that simple!

The other features from our release notes are:

  • Zip attachments while migrating
  • Support for Pass-through HTML
  • Special handling of documents containing just one attachment
  • Full folder migration (example: Domino.Doc binders)
  • Support for multiple Notes passwords
  • Improved lookup field support (add missing choices)
  • Improved support for migrating to Claims-based environments
  • Render with form improvements (computed subforms, etc.)
  • Discovery process is more robust

Some of these features will be more self-explanatory than others.  I will be posting detailed walkthroughs of a lot of these features here in coming days.  You can also check out the “what’s new” section of the user guide or watch my recorded “update” training here: http://communities.quest.com/docs/DOC-14812