Notes Migrator for SharePoint 6.1 is a significant product release that pushes the product further in three important areas: Design Migration, Content Migration and Pre-Migration Analysis. This release will be “Generally Available” on the Quest web site in a few days. If you were in the beta program you already have the final build (6.1.0.626). Below is a high level list of this release’s most important new features.
Design Migration
Migrate Notes Views to SharePoint Views – This feature will allow you to select a Notes view and generate a similar view in any SharePoint list or library. Because the mapping of columns in your view is intimately tied to the mapping and provisioning of data columns that occurs in your migration job, this new capability is also tied to your migration job. Specifically, view migration is found on the new Advanced tab of your Target Data Definition. You can design new views there and the views are provisioned when the migration job is run. A nice side effect of this model is that you can run the same migration job against many new or existing lists or libraries.
Press “Import from Notes” and then select the Notes view you want to migrate. The view migration wizard will then attempt to design an equivalent SharePoint view, mapping as many Notes columns as possible to existing SharePoint columns OR to new columns defined in your job’s target data definition. As you might expect, there are a number of things that can go wrong with such an automated mapping. The two most likely issues are (1) the Notes data item shown in the view had not been migrated to the SharePoint list yet or (2) the Notes view column is computed and the formula is too complicated for a tool to convert. For issues like this, manual intervention is required and the migration wizard guides you through that. In the View Columns step, it gives you a side by side view of the Notes and SharePoint columns and highlights the parts that it needs help with in red. You can decide to manually map a Notes existing columns, define a new data column (which will be added to your migration job), or even specify a formula for a new computed column in SharePoint. Similarly you can also specify the sort order, the grouping and the document selection rules, or you can just take the defaults.
Content Type Generation – These feature takes the tool’s ability to migrate schema (select fields from a custom Notes application and provision a similar schema in a custom list) to a new level. Now you can migrate your custom Notes application schema to SharePoint content types instead. You can create new Content Type Definitions on the new Content Types tab of your Database or Technical Class records in the Migration Console. You can design them from scratch or, more likely, you will generate the Content Type definitions from a Notes Form, from an existing migration job, or from an existing list that you have been working on. (The last two options allow you to do your initial development and testing on a custom list and then transition to content types later.)
Once you have generated an initial Content Type Definition, you can further customize it, specifying the parent content type, the group, and the columns. The columns editor looks like the Target Data Definition columns editor, but in this case you are defining Site Columns instead of List columns. When you are ready to provision your new Content Type, the tool will try to find existing Site Columns that match your specification and will provision new ones if needed.
New Column Provisioning Options – The ability to provision lists and libraries while migrating content has long been a popular feature. This capability has been greatly expanded to allow provisioning a nearly complete list of SharePoint column options, such as default values and data validation. You can even set columns to be read-only (something you can’t normally do in the SharePoint user interface). Note that as before, the provisioning options only apply when the tool is first provisioning the columns; it does not upgrade existing columns.
Provision Calculated columns – You can now define Calculated columns in your target data definition. These will be provisioned in SharePoint when your migration jobs are run. This Microsoft documentation page explains the legal syntax for SharePoint formulas: http://msdn.microsoft.com/en-us/library/bb862071.aspx. You can also use our new built-in formula editor, which appears at several points in the product.
Provision Lookup Fields – Lookup fields can now be provisioned in SharePoint lists. Previous versions of the tool allowed you to migrate to Lookup fields, but you first had to manually configure them in SharePoint. You can even configure lookups from lists in other sites in the same site collection (something you can’t normally do in the SharePoint user interface).
Content Migration
New Run Job options – During normal migration jobs, Notes Migrator for SharePoint first takes care of the provisioning steps (creating the list/library if needed, adding the appropriate content types to the list, adding any missing columns if needed, and creating views, setting the list/library permissions) and then migrates the content. Now you can now choose to run just the provisioning bits without migrating the content. This will be useful in cases where you want to review the resulting list schema and perhaps make changes to the list settings before migrating content.
Also note that the above screen shots show off the tool’s new Run Job button with the drop down menu of various modes of running the tool, such as importing data to and importing data from intermediate XML files. Since this button is also available in the Migration Console, users can now leverage the intermediate file options from there (it was formerly available in the Designer Client only). Similar functionality is also available in the tool menus and the bulk Migrate To SharePoint task in the Migration Console.
Migrate by Form Used – When selecting records from your Notes database, you can now elect to query records by form. You can pick one or more Forms from the Notes database design or type them manually. This will limit the records selected to include only those data records that were flagged with the forms you selected (i.e., where the “Form” item was set to one of those form names).
If you specify forms in this way, the Select Items and Columns dialog will also display the data items defined on your selected forms. (Remember, however, that background agents, etc., may have also set data items on the Notes documents, so you may still need to look at the Sampled Items node to discover those.)
Improved Content Type Support in Migration Jobs – Notes Migrator for SharePoint already had pretty good support for dealing with content types while migrating content, but this release extends those capabilities and also makes it much easier to use. The new Manage Content Types section on the first tab of the Target Data Definition allows you to push site content types into a new or existing SharePoint list. Adding the content type to the list is a prerequisite to writing documents with those content types, so this new feature eliminates that manual step and make more automation possible.
In addition, you can now associate each content type with one or more Notes forms. If you do this, then the tool will automatically assign the content type for each migrated document without the need to do any explicit mappings. (For more complex scenarios you can still explicitly map any piece Notes data, including the result of a formula evaluation, to a ContentType field.)
Note that this feature is completely independent of the ability to generate new Site Content Types, as described above under Design Migration, but you may often find yourself using the two in combination.
Normalize multi-valued items to multiple documents – It is common for Notes forms to implement detail records (the things that relational developers would have set up a one-many-relationship for) using multi-valued data fields arranged to look like a table, as shown below. Users would then fill in as many entries as the needed, making sure that the values from each multi-valued item lined up properly.
Now you can use Notes Migrator for SharePoint to extract the multi-valued items as separate records. To do this set the Multi-Value disposition option for the columns that you expect to contain arrays (ContactName, ContactTitle, and ContactPhone in the above example) to “Normalize”. This will cause NMSP to generate multiple SharePoint items (three in the above example) for the values in the columns designated for Normalization.
Note that if you also wanted “header” records you could first migrate those using a separate migration job. You could use lookup fields or some other mechanism to relate the “header” and “detail” columns. In the above example, Customer ID in the Contacts list is set up as a lookup field that references the Customers list (where Customer Name and Category are also stored).
Finally, you may encounter Notes forms that list out each item in separate rows, instead of using multi-valued items. NMSP can handle that case as well, but the migration team will need to do a little work with formulas to generate multi-valued columns to Normalize on. For example, you could define a source data definition column with the formula “Product_1: Product_2: Product_3: Product_4” to generate a “Products” array.
Dynamic link tracking for URL columns – When migrating Notes data to SharePoint URL fields, you can now leverage the tool’s popular Link Tracking Service for those links (the same way you currently do for Doc Links in Rich Text Fields). You do not need to do anything special to enable this. Simply start migrating data items that store notes:// links (or formulas that generate notes:// links) and they will be converted to dynamic links that will ultimately point users to the migrated versions of those documents.
As a reminder, the tool supports two input formats for Url fields.
- <url> - sets the url to be the display name in SharePoint
- <url>, < name> - sets the url and display name separately in SharePoint
Improvements in “Render With Form” function – This very popular feature, which allows you to extract the content of any custom Notes database (regardless of the complexity) so you can archive them as simple rich text documents in SharePoint, has been improved in a number of respects. Handling of dynamic subforms, computed fields, computed text, and keyword fields have been improved, resulting in even better looking rendered documents.
Provision Alternate Libraries – Users have often relied on the tool’s unique ability to migrate images and attachments to a separate location from the main documents being migrated. For example, you might want to place all attachments in the Shared Documents or Site Assets document library. Previously you would have had to provision these alternate libraries manually before running your migration job. Now the tool provisions these automatically if needed.
Analysis / Migration Console
Capture Design Copies during Analysis – Now when you perform a Design Analysis on a set of databases you can elect to also create a local design copy. For every database scanned, the tool will create a small copy of the database (design elements only, similar to a design template) on the local machine on which the design analysis is being performed. The intent of this feature is to allow for consultants and other migration personnel to be able to view the full database designs even while they are disconnected from the production environment, which is particularly important on large analysis projects where in-depth manual design analysis is required. The location of these databases is controlled by a setting on the Analysis tab of your Global Options dialog, and defaults to a folder in the tool’s <ProgramData> area. Users can the easily open the Local Design Copy from any database view.
Classify by Last Used (All Replicas) – This enhancement expands the available options for automatically determining which Technical Class or Business class a databases belongs to. Now you create a rule for recognizing class members based on when it was last used across all known replicas. For example, you can create a class that groups all databases with a Last Used date greater than 356 days ago.
Import data into repository from CSV files – This enhancement allows users to read in records from a CSV file and update vales in certain Quest Repository database records. The tool is available on the main Notes Migrator for SharePoint node. First the user is prompted to select a CSV file and then a mapping dialog is displayed. For each column in the CSV file, the user can choose to map it to one of the available database record properties. Not all database properties can be imported from external data sources. The intention here is that users who perform manual analysis/triage projects using spreadsheets or external tools can import that data back into the Migration Console.
This feature can also be used to add new database records into the repository (by Server and File Path, Replica ID, or Database Key). The intention here is to allow users to import a list of databases to be analyzed or migrated. Users would typically follow up such an import with a full analysis of those databases to populate the remaining database properties.
General
Performance and scalability for Migration Console – The database views have been re-architected to support 60,000+ databases at a time. With a very large repository, users may experience a delay while starting the tool as all the records are loaded into memory. After that, scrolling, sorting and filtering database views and opening additional views should be very fast. Special protections have also been added to prevent you from running very large reports that are likely to crash the migration console.
New User/Group Mapping options – When performing user/group name mapping, the Output Translation option now allows you to further transform the name that results from your name lookup before submitting that name for resolution in SharePoint. This may be useful when your environment requires specific name formats that are not immediately available in your mapping source.
Also, the Test User/Group Mapping tool now has a Validate in SharePoint button which will try to resolve the name that your configured mapping process produces so you can verify that it really works in SharePoint. This should make experimenting with and debugging various user mapping options a little easier for everyone. Note that this capability is only available when using client-side user mapping (not when configuring server-side mapping in the Import Service).
Improved CBA/FBA support – Connections to SharePoint sites using Forms Based Authentication now automatically renew cookies as needed during long migration jobs and other operations. The tool will routinely check the expiration time of the authentication tokens that it holds for the client and, if a token is due expire within a certain window, it will force a new authentication. (The time limit will be 30 minutes, but this can be changed in your SharePoint Environment settings.) Depending on your particular authentication system, this may appear to the user as a browser prompt forcing the user to re-authenticate. In other cases, it may appear as a browser window that opens briefly and then closes again.
Windows Authentication using alternate account for Link Tracking Database – One new connection option is available for connection to the Link Tracking Database. You can now specify that you want to use Windows Authentication, but supply an account other than your own.
Bulk Editing of certain database / class / job properties – Expanding on the tool’s capability to select multiple databases and set properties in bulk, you can now set a number of additional properties in a large number of selected databases. You can even set certain properties inside the migration jobs assigned to the selected databases. Finally, you now have a similar set of options for Technical Classes and Business Classes as well.
Usability improvements – A number of things have changed in the product to improve to make the tool easier to use, including rearrangement of dialogs and additions of menus. More pop-up help icons and more context sensitive links to help topics have been added. A completely rebuilt “Add/Remove Columns” dialog and the built-in documentation describing the 170 possible built in columns makes customizing views much easier.
