Lotus Notes to SharePoint Blog

Blog about Dell's Notes Migrator to SharePoint tool and other things related to Lotus Notes migration projects

Writing more than 2000 Notes documents to a SharePoint 2007 list or library

It is fairly common knowledge now that SharePoint 2007 lists and libraries can hold millions of documents, but there is a practical limit of about 2000 documents in any one view or folder.  Such limitations are discussed extensively in this Microsoft article [link].

So how do you deal with this when migrating Notes documents to SharePoint? 

Notes Migrator for SharePoint customers are certainly subject to the same per-folder limitations.  Our tool is very good, however, at assigning folder names dynamically so the problem is mitigated to a significant extent. 

For example one, could map the Category field or Territory field in Notes to the Folder name in SharePoint.  If these values contain slashes (“/” or “\”) you can use these to generate nested trees of folders and sub-folders.  For example map “US/Massachusetts/Newburyport” to a Folder name to get three levels of subfolders.

image image

You can even use formulas to generate folder names dynamically.  For example, use the following formula to generate three levels of folders: 

Country + “/” + State + “/” + @Uppercase(IndustyCode)

image

Some customers have gotten pretty creative in creating “randomized” distributions for folder names.  In this case, the folder hierarchy is not likely to be a good way to for users navigate documents, as the names are pretty much meaningless to them.  As with any huge list of documents, search becomes an important tool for users to search documents here.  You can also use tools such as Quest Web Parts for SharePoint to design much more performant views against large data sources.

Another option is, of course, to split the database into multiple smaller lists and libraries.  One good way to do this would be to use a selection formula to extract a portion of the data for a given job run:

Region = “NorthWest”

image

Breaking huge Notes databases into multiple lists and libraries can also helps with backups.  Again, you can use specialized web parts like Quest Web Parts for SharePoint‘s List View to create a single view of multiple libraries (even across multiple SharePoint sites).

What about SharePoint 2010?  Happily, the scalability limits are not nearly so severe here and Notes migrating teams so not have to worry about them nearly as much.  The new limits are described [here]. 

2 responses to “Writing more than 2000 Notes documents to a SharePoint 2007 list or library

  1. pratima31 April 10, 2012 at 3:19 am

    Nice post steve.
    Is it possible to migrate LN folder with all of its subfolders with a single Quest job? If there is a folder with 100 subfolders, at around 1000 documents all together, then is it necessary to run 100 jobs or does Quest provides a way to migrate all the subfolders using a single job?

    • swalch April 10, 2012 at 8:17 am

      We can migrate TO multiple folders in one migration job. You just map whatever data item to want to a Folder column and that becomes the new folder name in SharePoint. The folders get created automatically.

      We many people want to do is to extract the old Notes folder names and use that as the basis for assigning folder names in SharePoint. This is hard for two reasons:
      (1). Most Notes Documents do not store the old folder names as data items. If you are a clever Notes developer, you may find a way to do that, perhaps by writing some LotusScript to preproces the documents. More on that in a future post.
      (2). In Notes, one document can be in multiple folders but not in SharePoint. So you may need to think though how you want to resolve that.