In an attempt to leverage all the benefits of Office 365 and accelerate the retirement of classic/legacy Windows fileshares (Windows Server 2003 fileshares nonetheless), we simply encouraged our client to buy additional OneDrive storage and allow us to push the data up there. Once we are off those unsupported servers, we will come back and clean up the content.
Not so fast.
While SharePoint Online/OneDrive is a fantastic platform, in most customer cases, it is a shared platform. One must forgive Microsoft for enforcing certain limitations to protect customers from each other and from themselves. We migrated 7 TBs worth of data to OneDrive using Sharegate and PowerShell, then found ourselves in a pickle trying to cleanup that data, since even PowerShell has its limitations when working with SharePoint Online.
I advocate moving to the cloud, but it is no secret that as far as control is concerned, on-premises is still king. Using Windows Scripting Host and the FileSystemObject, WinDirStat, Filewise and the plethora of other tools available, is a preferred approach to file cleanup and archiving, prior to migrating to the cloud.
The metaphor is simple. If you are moving into a new house, you are better off getting rid of what you don’t need prior to the movers getting there.
The 5000 item limit threshold in SharePoint makes it close to impossible to manage extremely large document libraries, and that’s even when using PowerShell. Performance when working across the wire is not optimal, and even though you can improve upon that a bit, using Azure, it pales in comparison to working in a LAN environment.
Finally, PowerShell memory management is not as mature as one would hope and in certain situations, when you can’t dispose of variables, the script can end up consuming gigs and gigs or RAM, thus bringing the host to its knees and slowing the process down even more.
Get your customers to Office 365 ASAP. Clean up first though. Files that were created in Excel 5.0 and last modified in 2001 are more than likely good enough to be archived, if not simply deleted.