Run migrations in advanced mode

Migrate for AODocs is a tool that allows you to migrate files and folders into your AODocs libraries.

Important: Migrate for AODocs is currently available for our service teams only. For EAP access, contact dev-migration@altirnao.com

This article explains the full, advanced options available to run your migrations.

Learn more about the simplest use case of importing a Google Drive folder into AODocs.

 Access advanced mode and choose the type of migration to run 
 Extract details of your source data and migration setup
 Configure common options for migrations
 Configure migrations from a local file server
 Configure migrations from a Google Drive folder
 Configure migrations from Lotus Notes
 Configure migrations from SharePoint
 Configure migrations “from scratch” using a Google spreadsheet
 Configure migrations “from scratch” using a CSV file
Run your migration

Access advanced mode and choose the type of migration to run

You're ready to run advanced migrations if:

1. In the main screen, press Advanced migrations or select Migrate into AODocs in the menu.

image01.png

2. Install Gateway if you're planning to run any of the following migration types: 

Important: These migration types are grayed out (unavailable) until you install Gateway.

3.  Press Create, then select the type of migration to run. A configuration screen opens.

Tip: When you have started a migration, you can monitor its progress in the Running tab.

image02.png


Extract details of your source data and migration setup

Before running any type of migration except “from scratch” migrations using a Google spreadsheet,  you can extract details of your source data and migration, according to the options you selected in the configuration screen. This gives you an idea of how to proceed when you run your migration.

Notes:
– You can’t run an extraction of manually-created Google spreadsheets
– Information about relations is not extracted, so isn't included in the extraction spreadsheet. Learn more: Configure relations for your migration.

To run an extraction:

1. Define your source data in the configuration screen:

Note: Optionally, you can:
select your AODocs library and document class – learn more: CLASS_NAME column
define how many versions of files to include (if your source data is versioned) – learn more: VERSION column

2. Choose the required format:

  • Preview in Sheets – an extraction spreadsheet opens in a new tab 
  • Preview in CSV – three CSV files are downloaded locally (in the folder where you ran Gateway) to a subfolder called migrations: binding.csv, data.csv, mapping.csv

Note: The extraction spreadsheet is identical to the migration spreadsheet, but you can’t use it for migration. The extraction and migration spreadsheets are generated in exactly the same way, according to the options you select in the configuration screen.


Define common options for migrations

The following options are used in several migration types. 

Under Configure the source

1. If required, filter the documents to include in the migration, by creation date and/or modification date.

Note: This option is available for file server, SharePoint and Lotus Notes migrations only. 

2. If required, in the Versions field select how many previous versions to include:

  • no versioning (default) 
  • 3
  • 5
  • 10
  • 100

Notes:
– Drive migrations don't have the versioning option: when you migrate from Dive, files retain their versioning history as attachments and not as versions of AODocs documents.
– Learn more about versioning for Google spreadsheet migrations and CSV migrations.
– The options Pattern matcher, Permissions and Relations aren't available in this version.

common_source.png

Under Define target options

1. Choose the target AODocs library: when you enter three or more digits, a drop-down menu appears with all the matching libraries in which you're a contributor.

2. If required, select the document class.

Notes:
– For SharePoint migrations, you usually don’t need to select a document class because the CLASS_NAME column in the spreadsheet is populated with SharePoint’s Content Type. However, you can select a document class: if there are any missing Content Type values in the source data or if you remove a value from a CLASS_NAME cell, the document will be imported into the class you selected in the configuration screen.
– For all other migration types: if you select a document class, the spreadsheet doesn't contain a CLASS_NAME column (all documents will be imported into the document class you selected). If you don’t select a document class, the CLASS_NAME column in your migration spreadsheet is empty and you must fill it in manually.
– Learn more: CLASS_NAME column.

3. Optionally, select a template.

Note: If templates are defined in your document class, you can select from the list of available templates or select blank to select the document class without a template.
Learn more:
Handle templates in your migration sheet
Create and configure document templates

4. Define the type of storage for your migrated files:

  • Google Drive: your files are migrated to Google Drive and become attachments of AODocs documents
  • Google Cloud Storage (GCS): your files are migrated into your GCS storage bucket and referenced as a link in the AODocs document

Note: Access to GCS attachments uses the same authorization as the AODocs document.

5. For Google Drive and Google Sheets migrations, define whether to:

  • Move all files (recommended)
  • Move files (and copy when moving is not allowed)
  • Copy all files

Notes:
– This menu is unavailable if you selected GCS in the previous step; you can't move files from GCS.
– When you move files, they keep their Google Drive URL.
– Copied files always have a new Google Drive URL.
Move all files isn't available for super administrators.

6. Select which (if any) Microsoft Office files to convert to Google format.

Note: You can convert XLS, XLSX, DOC and DOCX files.

common_target.png

Under Wrap-up

1. Check the summary. 

Tip: If required, you can follow the link to access your AODocs library.

2. Enter a name for your migration so it is easily identifiable in the migration panel. If you don’t enter a name, the last five characters of the identifier are used as the name. 

3. Define the type of storage for your migration spreadsheet:

  • Google sheet: a Google spreadsheet is stored in docs.google.com, visible in Google Drive
  • Google Cloud Storage (GCS): select this option if you have more than 5 million items of data (this means your spreadsheet would contain more than 5 million cells, which is the limit of Google spreadsheets). The file format is CSV, but you  can't access it before running your migration.
  • Google Drive: a CSV file is stored in a folder called Migration in Google Drive

4. Select Edit sheet before import to confirm manually at the upload and setup AODocs stages of your migration. If you want your migration to proceed without manual validation at these stages, leave the checkbox unselected. Learn more: What is the Migrate for AODocs process? 

Note:
– If you selected Google Cloud storage in the previous step, this option isn't available.
– If you want to define relations, select this option so you can add information about relations in your spreadsheet before migrating.
– For "from scratch" migrations, manual validation is always required at the upload and setup AODocs stages regardless of whether you select the Edit sheet before checkbox.

5. You can select Enforce Gateway processing for full cloud migrations (such as SharePoint online or Google Drive) if you're running a huge migration and you faced some issues running it in full cloud.

common_wrapup.png


Configure migrations from a local file server

After accessing advanced mode and choosing to create a  File server migration, the configuration screen opens.

1. Under Configure the source, enter the absolute path of the local folder you want to migrate.

Tip: Press the folder icon to select your path.

Note: If you enter a folder that doesn’t exist, an error message appears.

2. If required, unselect Skip hidden and system files (selected by default).

Note: For most file server migrations, there’s no versioning. However, if you’re migrating from a Windows server with versioning, you can select how many previous file versions to migrate in the drop-down menu.

3. Fill in the common options for migrations under Configure the source.

4. Press Next.

FSimage01.png

5. Fill in the common options for migrations under Define target options, then press Next.

6. Fill in the common options for migrations under Wrap-up.

You're now ready to run your migration.


Configure migrations from a Google Drive folder

Important: You can't migrate files that are shared with you in Google Drive. You must own all the files you migrate.

After accessing advanced mode and choosing to create a  Google Drive migration, the configuration screen opens.

1. Under Configure the source, select a folder in Google Drive or in a shared drive (left field), or an entire shared drive (right field), then press Select.

Note: To move files from a shared drive, you must be a Manager or Content Manager in the shared drive. To copy files from a shared drive, you can be a user in the shared drive.

Tip: Click the Google Drive icon in the left field to open the Google Drive or shared drive folder. You can recover the Drive ID of your selected folder if required.

Googleimage01.png

2. Fill in the common options for migrations under Define target options, then press Next.

3. Fill in the common options for migrations under Wrap-up.

You're now ready to run your migration.


Configure migrations from Lotus Notes

Note: Before running a migration from Lotus Notes, you must export your data from Lotus Notes. A local folder is created, which you then import into AODocs using Migrate for AODocs.

After accessing advanced mode and choosing to create a  Lotus Notes migration, the configuration screen opens.

1. Enter the local path of the Lotus Notes folder you want to export. The list of forms in your selected folder is displayed, with the number of documents in each form.

Note: Forms in Lotus Notes are the equivalent to document classes in AODocs.

2. If required, select the forms you want to export. By default, no forms are selected  and they are all exported. 

Tip: If you have a long list, you can enter text in the Name field to filter your list.

3. Press Next.

Lotusimage01.png

4. Fill in the common options for migrations under Configure the source, then press Next.

5. Fill in the common options for migrations under Define target options, then press Next.

6. Fill in the common options for migrations under Wrap-up.

You're now ready to run your migration.


Configure migrations from SharePoint

After accessing advanced mode and choosing to create a  SharePoint migration, the configuration screen opens.

1. Under Configure the source:

  • in the SharePoint type field, select:
    • Online if you’re using online SharePoint
    • SharePoint 2013+  if you're using SharePoint 2013 or above
    • SharePoint 2010- if you're using SharePoint 2013 or below
  • enter your SharePoint URL
  • enter your SharePoint username and password

2. Press Check.

SPimage01.png

3. If required, press Extract a preview of libraries in a spreadsheet to generate a spreadsheet listing the SharePoint sites and lists you have access to (depending on the SharePoint URL you entered).

Tip: Press the Sharepoint URL to open SharePoint in a new tab.

SPimage02.png

The SharePoint structure spreadsheet contains the following information:

image04.png

4. Select the required SharePoint site from the drop-down list, which lists all sites available from the SharePoint URL you entered. 

Note: If your SharePoint URL contains only one site, it is selected automatically. 

5. Select one or more SharePoint lists.

Tip: Enter a string in the Name field to filter the list.

6. Fill in the common options for migrations under Configure the source.

Note: If you don’t use versioning in SharePoint, leave the default option no versioning selected. Learn more: VERSION column.

7. Press Next.

SPimage03.png

8. Fill in the common options for migrations under Define target options, then press Next.

9. Under Wrap-up, choose whether to enable the cache:

  • Cache enabled to optimize the SharePoint throttling (default): when documents are being created in AODocs, the responses from SharePoint are stored in cache. This avoids having to make multiple calls to SharePoint, so the migration process is faster. However, if you run incremental SharePoint migrations you must clear the cache manually to be able to identify which source files have been changed since you ran the last migration.
  • Cache disabled, only use it for debug: when documents are being created in AODocs, the responses from SharePoint aren’t stored in cache, so multiple calls to SharePoint are required. This makes the migration process slower. However, if you run incremental SharePoint migrations there’s no need to clear the cache manually.

Learn more: What is throttling?

10. Optionally, before starting your migration you can flush the Gateway cache from previous migrations. This is useful if you recently made changes to your SharePoint files or folders.

Note: You must have Gateway connected to use the Flush cache option.

11. Fill in the common options for migrations under Wrap-up.

You're now ready to run your migration.

SPimage04.png


Configure migrations “from scratch” using a Google spreadsheet

You can run migrations from a blank Google spreadsheet that you fill in from scratch. This feature is usually used to create new documents in AODocs in contexts where you don’t have any source files.

Note: You can’t run an extraction of a manually-created Google spreadsheet.

Generate your migration spreadsheet

After accessing advanced mode and choosing to create a  Google Sheets migration, the configuration screen opens.

1. Select:

  • With versions: up to 100 versions are allowed and each row represents a document version 
  • Without versions: versioning is disabled and each row represents a document

2. Select:

  • No attachments
  • Local attachments – you can add files from the following locations: 
      • drive:/[Google Drive ID]
      • gs:/[storage bucket name]/[filename]
      • file:///C:[folder structure]:[filename]
  • Cloud attachments – you can add files from the following locations: 
      • drive:/[Google Drive ID]
      • gs:/[storage bucket name]/[filename]

3. Press Next.

GSimage01.png

4. Fill in the common options for migrations under Define target options.

5. Fill in the common options for migrations under Wrap-up.

6. Press Start Migration

Fill in your migration spreadsheet

1. Open the migration spreadsheet from the card that appears on the Migrate for AODocs screen.

2. Enter a different ID for each document you want to create.

Note: These IDs aren’t used as IDs for your target files. In the context of a “from scratch” migration, they’re used to indicate to Migrate for AODocs to create a different document for each ID.

3. Enter a title for each document.

4. Enter an initial author for each document.

5. In the data sheet, create or add values to any required columns.

In this example we created:

  • the CLASS_NAME column
  • the VERSION column
  • columns for properties Contract No. and Amount

Note: If you selected a document class in the configuration screen, columns are created in your spreadsheet for all existing properties in the document class you selected. You can add values as required. Learn more: The data sheet.

image05.png

You can add or edit any of the standard columns in the data sheet:

  • VERSION – enter different version numbers for two or more identical IDs; the format must be X[.Y[.Z]] (where X, Y, Z are integers)
  • VERSION_DESCRIPTION
  • FOLDER
  • RICH_TEXT
  • CREATION_DATE
  • MODIFICATION_DATE
  • UPDATE_AUTHOR
  • ATTACHMENT -– enter the path of a file, for example:
      • local file:
        file:///Users/carolinecharron/Documents/Migration3/AllContracts2019.xls
      • Google Drive file:
        drive:/1veHwMzLy_hgU0iXOJruZdQMuKv_7CEGsPfL6vJAeXdk
      • GCS file:
        gs:/MIGRATION_STORAGE_BUCKET/TrainingPlan.docx
  • [Property names] – enter values in columns corresponding to properties in the document class you selected or create new columns to create new properties.

6. In the binding sheet, set up binding between any columns from the data sheet you want to include in your migration and your AODocs configuration.

7. In the mapping sheet, map the users you entered in the data sheet to users on your AODocs domain.

Continue your migration to create documents in AODocs

1. Refresh your migration panel. While you have been filling in your migration sheet, your migration has passed through the extraction phase and is now in the upload phase. 

Learn more: What is Migrate for AODocs?

2. Add any required attachments.

3. Press Click to start upload in Gateway. Migrate to AODocs uploads any attachments you added and your migration proceeds to the validation phase. 

Continue with your migration. Learn more: Run your migration.


Configure migrations “from scratch” using a CSV file

You can run migrations from a blank CSV files that you fill in from scratch. This feature is usually used to create new documents in AODocs in contexts where you don’t have any source files.

Generate your migration spreadsheet

After accessing advanced mode and choosing to create a  CSV migration, the configuration screen opens.

1. Enter the absolute path of the local folder you want to migrate.

It must contain only the CSV files that contain the details of your migration: 

Tip: Press the folder icon to select your path.

Notes: Error messages appear if:
– you enter a folder that doesn’t exist
– your CSV file contains semi-colons instead of commas  

CSVimage01.png

2. Fill in the common options for migrations under Define target options.

3. Fill in the common options for migrations under Wrap-up.

4. Press Start migration.

5. Open the migration spreadsheet from the card that appears on the Migrate for AODocs screen.

6. Fill in your migration CSV file in the same way as for Google spreadsheets.

7. Continue your migration to create documents in AODocs in the same way as for Google spreadsheets.


Run your migration

After configuring your migration:

1. Press Start migration to create a migration spreadsheet (extraction phase).

There are two scenarios: 

  • You didn't select the Edit sheet before import checkbox and you're not running a "from scratch" migration: your migration proceeds directly through all the stages without user validation. Learn more: What is Migrate for AODocs?
  • You selected the Edit sheet before import checkbox (learn more: common options for migrations under Wrap-up) or you're running a "from scratch" migration: your migration moves to the Upload files column and is paused. You must confirm manually at the upload and AODocs setup stages to proceed with your migration (as described in the reminder of this procedure).

Note: If there are any errors, your migration stays in the current column and an error message is displayed. 

2. When your migration is paused in the Upload files column, you can open the migration spreadsheet and make any required changes to it.

For example, if you want to include relations between your documents when you migrate, manually add the required information in the migration sheet. Learn more: Configure relations for your migration.

3. Press Next stage to start the upload.

Important: Once the upload has started, don’t change the attachments in the migration spreadsheet.

4. When your migration is in the Setup AODocs column, press Next stage. Migrate for AODocs checks that your migration is valid.

5. Press Next stage again. Migrate for AODocs starts creating your files in AODocs. When the migration is complete, it disappears from the Running tab.

Learn more:

Was this article helpful? 1 out of 1 found this helpful
If you didn’t find what you were looking for, don’t hesitate to leave a comment!
Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.