Special characters in CSV files for migrations "from scratch"

Migrate for AODocs is a tool that allows you to migrate files and folders into your AODocs libraries.

Important: Migrate for AODocs is currently available for our service teams only. For EAP access, contact dev-migration@altirnao.com

When you configure migrations “from scratch” using a CSV file, you must respect some conventions to match the requirements of the generated spreadsheet and the value generated in the target AODocs documents. 

Standard CSV conventions cover most cases, but we implemented additional special characters in Migrate for AODocs to cover the remaining cases. 

This article outlines conventions you must respect. 

Automatically generated table of contents


Important: In the CSV file, any special characters must be escaped to prevent incorrect content. It may be necessary to escape once or twice, depending on the storage type, CSV (stored in Google Cloud Storage or Google Drive) or Google Sheets. For example, if the migration is based on a CSV file and is also stored as CSV, you need to double escape: the first time to escape the content that will be extracted from the CSV file; the second time to escape the value stored in the CSV file generated by your CSV file. 

Standard CSV conventions

CSV value escaper: double quote

  • Columns affected: all
  • Use a double quote before and after any value containing a space, a comma or any special character, to escape these characters.
  • If a double quote is present in a value, always escape it using the backslash character before the double quote. In addition, the whole value must be surrounded by double quote characters.
  • Examples (a full row):
    • A,B,C is equivalent to “A”,”B”,”C” leading to A and B and C as values in AODocs
    • “A”,”B\”C”,”D” leads to A and B”C and D as values in AODocs

CSV special character: comma ,

  • Columns affected: all
  • The comma is the reserved character used to split the line into multiple properties.
  • If a comma is present in a value, the value must be surrounded by double quote.

CSV character escaper: backslash \

  • Columns affected: all
  • The backslash is used to escape special characters in a CSV file.
  • Example for any special character:
    • if the CSV file is stored as Google Sheets: “company\”Subsidiary\””
    • if the CSV file is stored as CSV: “company\\”Subsidiary\\””
  • If a backslash is present in a value, escape it using a backslash before the backslash character to retain:
    • if the CSV file is stored as Google Sheets: “company\\Subsidiary”
    • if the CSV file is stored as a CSV: “company\\\\Subsidiary”

Migrate for AODocs conventions

Multivalue separator: pipe |

Note: Learn more: Wikepedia Pipe.

  • Columns affected:
    • properties that always support multivalue: ATTACHMENT, ATTACHMENT_NAME
    • properties for which multivalue has been enabled: STRING, NUMBER, URL, CATEGORY, REFERENCE_CATALOG, and so on.
  • The pipe is used to split a single aggregated value in the CSV file into multiple values into AODocs.
  • If the property is not multivalued, pipe will be read “as is”.
  • If a backslash is present in a multivalued property, it can be escaped with a backslash like other characters.
  • Examples:
    • Multivalued property:
      • A|B leads to A and B in AODocs
      • A\|B|C leads to A|B and C in AODocs
    • Non-multivalued property:
      • A|B is equal to A\|B and leads to A|B in AODocs

Multilevel separator: greater than >

Note: Learn more: Wikipedia Greater than sign.

  • Columns affected: CATEGORY
  • The greater than sign is used to split a single aggregated value in the CSV file into multiple levels into AODocs.
  • The multilevel separator is always processed after the multivalue separator.
  • If a greater than sign is present in a category, it can be escaped with a backslash like other characters.
  • Example:
    • Multivalued property :
      • A>B leads to B in A in AODocs
      • A\|B>C|D leads to C in A|B and D in AODocs
      • A>B|C|D leads to B in A and C and D in AODocs
    • Non-multivalued property:
      • A>B|C|D leads to B|C|D in A in AODocs

Unsupported URI characters

Learn more: URI in the RFC.

CSV special character: space

  • Columns affected: all
  • Values are trimmed before being processed. Then, after splitting by value and level if applicable, any spaces at the beginning and end of a value are removed.
  • Examples:
    • A or  A or A  leads to A in AODocs
    • A|B or A | B  leads to A and B in AODocs
  • The space is not compatible with the ATTACHMENT column as it isn't a URL-compatible parameter. You must encode it using %20

Norms

URI

  • Columns affected: ATTACHMENT, URL
  • Accepted characters:
    • upper case letters: ABCDEFGHIJKLMNOPQRSTUVWXYZ
    • lower case letters: abcdefghijklmnopqrstuvwxyz
    • numerals: 0123456789
  • Any other characters must be encoded. Here is a list of the most common: URL Encoding of Special Characters.

Dates

  • 2021-11-18T13:00:00 ⇒ 2021-06-18T13:00 (UTC)
  • 2021-06-18T13:00:00 ⇒ 2021-06-18T13:00 (UTC)
  • 2021-11-18T13:00:00[Europe/Paris] ⇒ 2021-06-18T12:00 (UTC)
  • 2021-06-18T13:00:00[Europe/Paris] ⇒ 2021-06-18T11:00 (UTC)
  • 2021-11-18T13:00:00[Europe/London] ⇒ 2021-06-18T13:00 (UTC)
  • 2021-06-18T13:00:00[Europe/London] ⇒ 2021-06-18T12:00 (UTC)

Learn more:

Was this article helpful? 1 out of 1 found this helpful
If you didn’t find what you were looking for, don’t hesitate to leave a comment!
Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.