Notes SharePoint Blog

Steve Walch's blog about his favorite migration tool and other things related to Lotus Notes migration projects

Tag Archives: SharePoint

What data lives in “social software” systems?

When writing a world-class social data migration tool, we think a lot about what kind of data lives in these systems.  In particular, what data can reasonably be normalized and migrated between different systems. 

Here is our working list:

  • User Profiles
  • Blogs, Micro-blogs, Ideation Blogs, Wikis
  • Documents, Files, Folders, Libraries
  • Activities, Task Lists, Calendars
  • Discussion Forums
  • Bookmarks, Tags, Recommendations, “Likes”
  • Content ratings, # Visits, Badges earned
  • Social network (friends and coworkers)
  • Newsfeeds, Activity streams
  • Communities

That’s not to say that mapping data between disparate systems is easy!  Each of these content types have a lot of detail behind them, including different properties, categorization metadata, authoring metadata, cross-referencing systems, and rules for how rich text and attachments are structured.  Sometimes there is a huge mismatch between the fundamental models.  (For example, the file library model is quite different between SharePoint and IBM/Lotus connections.)  But that’s what makes migration tools fun!

Comparing Notes Migrator for SharePoint with Social Migrator for SharePoint

If you are familiar with Notes Migrator for SharePoint, the following table might help you understand what Social Migrator for SharePoint does and how it is structured.  As you can see, we have made some very different design choices…

 

Notes Migrator for SharePoint 6.2

Social Migrator for SharePoint 1.0

Sources

Lotus Notes, QuickPlace, QuickR, Domino.doc

IBM/Lotus Connections 3.x.  Many others on the roadmap!

Targets

SharePoint 2007, 2010 (and soon 2013), including Office 365.  Also SQL Server and export to file system.

SharePoint 2010 and 2013, including Office 365.

Connections to SharePoint

Standard SharePoint Web Services or Quest Import Service

Standard SharePoint Web Services only

Standard migration jobs

All standard application templates (discussion, document library, calendar, team room, etc.)

All service/application content types (blogs, wikis, activities, communities, files, forums, etc.)

Custom migration jobs

Yes. In fact, the main focus of the tool is our ability to deal with content from ANY custom application.  Thus the dazzling array of advanced features, including custom field mapping, formulas, and document generation!

No.  We are trying to keep it streamlined and simple here.  Connections has very structured content types and we tried hard to make the best choices without users having to think about mapping options, etc.

Supports migrating rich text, attachments, images, doc links

Yes

Yes

Migrate content to SharePoint profiles

No (at least not yet)

Yes, Connections profiles map to SharePoint profiles (including photos and friends).

Licensing options

Per Notes database, or per managed user (only for larger environments).

Per managed user only.

What is happening in the Social Software for the Enterprise?

My eyes were really opened when I attended the YamJam conference last October.  I have been building “collaborative business applications” since 1993 when Lotus was defining the category with Notes 3.0, so I definitely get the value of social software.  But when I saw the level of application integration that people were now achieving (for example, the ability to @mention a specific sales opportunity or “follow” a helpdesk ticket), I saw that ad-hoc collaboration was reaching a new level.

So here is how I now describe the “Business Social” or “Social Software for the Enterprise” category to people:  Imagine your company had a private instance of Facebook, LinkedIn, Twitter or Google+.  Now you can do things like…

  • Work in structured or ad hoc teams and communities
  • Follow and share new ideas and assets
  • Create a “social graph” of who knows who and who is interested in what
  • Help global organizations feel more connected and engaged
  • Transform static intranets into vibrant dynamic places
  • Stay updated on current projects and team activity
  • Integrate social features into enterprise applications

Last quarter, Gartner’s added a few new entries to their Magic Quadrant for Social Software for the Workplace in October (link).  The “Leaders” list to now includes Jive, IBM (Connections), Microsoft (SharePoint), Yammer (Microsoft), and Salesforce.com (Chatter).  With all these major players, it is no surprise that some of our biggest customers are already interested in migrating between them. 

In particular, a surprising number organizations who were early adopters of platforms like Connections and Jive now want to migrate to SharePoint.  They now see SharePoint (especially SharePoint 2013) as being “ready for prime time” as a social platform and they would much prefer to consolidate on a Microsoft solution.  This of course is why we built Social Migrator for SharePoint!

New tool for migrating IBM/Lotus Connections to SharePoint

So why has this blog been so silent these past few months?  Well in addition to general busyness, some extended vacation time, and, oh yeah, being acquired by Dell, much of my energy and attention has been going into a top secret new product.  And now, at last, I can talk about it…

Introducing Social Migrator for SharePoint!

Social Migrator for SharePoint migrates “social” content from IBM Connections (formerly known as Lotus Connections) to Microsoft SharePoint 2010, SharePoint 2013, and Office 365.  This includes content such as Profiles, Blogs, Wikis, Activities, Forums, File Folders, Bookmarks and Communities.  All content will be mapped to the logical SharePoint targets with an emphasis on automation and extreme ease of use.

For a preview of what we are doing in this space, please check out our recorded webcast here

We have been piloting this new tool with a global insurance company migrating 40,000 Connections users.  We plan to ship this quarter, but if you can’t wait, please do contact us to learn more and get a pilot project going in your environment.

Subsequent versions will target other “social” data source and target platforms.  So if you have built up significant business assets in Jive, Yammer, etc., please pass your requirements along to me.

Why is this space so hot right now?

Someone inside Quest recently asked me why the Notes application market is so white hot right now.  Thought I would share my answers here…

1. This is prime time for Notes migrations.  SharePoint 2010 and SharePoint Online have opened the floodgates of a migration wave that was already peeking.  Office 365 in particular delivers many of the improvements that Notes customers were eagerly waiting for.  Hundreds of the world’s largest organizations are committed to migration off of Notes over the next few years.  Similarly, many of the world’s largest system integrators have geared up to help them.

2. SharePoint 2010 and SharePoint Online create many new opportunities – and new challenges – for migrating complex Notes applications.  New capabilities such as document sets, managed metadata, search and scalability improvements, wiki pages, office integration are all seen as game changers for Notes shops looking to migrate.  InfoPath list forms, new built-in workflow and data validation features, and the ability to integrate external SQL Server databases as “external lists” dramatically reduce the cost of rebuilding complex applications.  Quest was the first Notes application migration tool to support SharePoint 2010 and SharePoint Online and (with Notes Migrator for SharePoint version 6.0) is rapidly innovating further ways to allow Notes migration customers to leverage these new platform capabilities. 

3. This is not a job for lightweight migration tools.  Because Notes was the platform of choice for building secure, collaborative applications for a so many years, organizations may have tens of thousands of old Notes applications.  Some are still business critical and many others contain valuable and often highly sensitive data that needs to be preserved with full fidelity.  Migrating all these applications correctly can be a very expensive undertaking for an IT department or an outside consultant.  Quest continues to invest heavily in this problem space because we realize that every incremental improvement we make can literally mean millions of dollars in ROI for our customer base.  It is paying off as the deeper people go into complex migrations, the better we look.

Recap of Domino.Doc migration features

We have had pretty darn good Domino.Doc migration features since the beginning, but until recently I have not blogged about it very much.  Now that we about to release support for migrating to SharePoint Document Sets in Notes Migrator for SharePoint 6.0, I thought it was time for a quick Domino.Doc feature roundup:

  • Notes Migrator for SharePoint gives you default data definitions for migrating the defaults
    • If you really use Domino.Doc to store attachments with some metadata, than SharePoint “document libraries” are a great choice
    • If you store rich text or multiple attachments in one “document” you might prefer to send it to a SharePoint list instead
  • In the cases where users have build custom “Document Types” we give you full power to migrate the custom fields, etc.
  • We allow you to map Document Types to SharePoint Content Types (but you have to design the target Content Types yourself in advance in the current version)
  • We allow you to map Binders to SharePoint Folders
  • We map access control rules at the Cabinet level and Document level
  • You can migrate complete version histories [link]
  • You can migrate unpublished documents
  • Our analysis tools discover the hierarchy of Libraries -> Cabinets -> Binders -> Documents (similar to what we do with QuickPlace)
    • Using the automation features, you can automatically provision sites/subsites for every Domino.Doc library/cabinet
    • You can also automatically generate a new SharePoint list/library on the same site for each Domino.Doc library/cabinet
  • Our analysis tools help you zero in on which Cabinets have been customized and what customization have occurred (new Doc Types, changed subforms)
  • Using Notes Migrator for SharePoint 6.0, you will be able to migrate to SharePoint 2010 Document Sets.  Either…
    • Map each Domino.Doc Binder to a Document Set and out all the documents inside it
    • Map each Domino.Doc Document to a Document Set and out all the attachments inside it

Putting all this together, we do a good job at automating the discovery, target assignment, site/library provisioning, and content migration.

Migrating Domino.Doc version histories (and unpublished documents) to SharePoint

Here is a screen shot of the new Domino.Doc selection options in Notes Migrator for SharePoint with the “versioning” parts highlighted in red:

image

For backward compatibility, the defaults are “Latest Major Versions” and “Published Documents” only, which migrate just one copy of each document.  However, you now have the ability to request prior versions and unpublished versions of each document as well.  This, of course, can result in multiple copies of each document.

image  image

In Notes Migrator for SharePoint terminology, these extra versions of the same document are treated as duplicates of the same document and thus our duplicate document handling options are triggered.  Our “If duplicate found” options were actually first implemented to handle incremental migrations (migrate the same database over and over and pick up any changes since the last time) but they turn out to be pretty darn useful for migrating version histories.

image

If you wanted to make each version appear as a different document in SharePoint, select “Write New Item”.  A more interesting choice, however, is to select “Create New Version”.  This will cause each version of your Domino.Doc document to become a new version of the same document in SharePoint. 

image

Note that you need to make sure that Versioning is enabled in your target SharePoint list or library:

image

One caveat here…   Notice that I did not migrate unpublished versions (for example, version 2.1 in the above document).  Unfortunately the current version of SharePoint does not allow us to explicitly set version numbers as we migrate.  So if I attempted to migrate versions 1.0, 2.0, 2.1 and 3.0 it would have ended up as 1.0, 2.0, 3.0 and 4.0.  Similarly, if my Domino.Doc library had pruned old versions and we now had only 3.0 and 4.0, that would have become 1.0 and 2.0 in SharePoint.  We are trying to convince the SharePoint product team to address this limitation for SharePoint 14, but for now keep in mind that this feature works best if you select all published versions in your query.  A workaround when you can’t do that is to explicitly migrate the “{VersionNumber}” field to an additional SharePoint meta-data field so you will always have that as a reference.

Formulas that deal with multi-valued items or multi-valued results

Everyone knows that Notes Migrator for SharePoint has the ability to run formulas (using the Lotus @Formula language) to transform data as you migrate it.  This feature is used for a variety of things ranging from concatenating two fields to translating status code field values to using an @DBLOOKUP to “join” data from a different view.  People are sometimes unclear, however, how they would use this facility to process multi-valued data.

There are two parts to this problem:

1.  How to create a formula that yields multi-values items:   

There are many constructs in the Notes formula language that give multi-valued items as results.  The simplest is to just use a colon “:” to concatenate text values into a text array.  Two examples are shown below:

    ‘A’ : ‘B’

    @Uppercase(Region) : @Lowercase(Region)

Also note that many @Formulas will yield a multi-valued result if the input was multi-valued to begin with.  For example, if Categories is a multi-valued item, then this formula will yield a multi-valued result:

    @Uppercase(Categories)

Be sure to specify the MULTI option in your Source Data Definition column so that the query engine extracts multi-valued data.

image

2.  How to map multi-valued columns to SharePoint fields:  

Only three field types in SharePoint allow multiple values (Choice, Lookup and User).  For these, you need to specify MultiValue in the appropriate Target Data Definition field for NMSP to attempt to write data that way.  (If you are writing to an existing target, you may need to modify the column properties in List Settings as well.) 

image

For other SharePoint field types (such as Text items) you have to make a choice as to whether you want to concatenate the multiple values into a single value, take just the first value, or take the last value.  You can control this option via your detailed mapping options.

image

Follow

Get every new post delivered to your Inbox.