Importers (File, SQL, Rest)

Modified on Fri, 15 Aug at 1:13 PM

TABLE OF CONTENTS



Introduction


A new suite of generic importers was introduced in v15, enabling data imports from various sources such as Excel files, SQL databases, and REST APIs. The common features shared by these importers are outlined below, while functionality specific to individual importers is documented on their respective help pages. 



Import Status

The table below provides information on the elements within this panel. This panel is common within all the importers.


UI ElementDescription
Run File ImportA button to run the File Import sync now.
Schedule File Import #1Specify the times to run the File Import.
Schedule File Import #2Specify the times to run the File Import.
Last Import DateDisplays the last time the File Import ran successfully. 
Current Task ProgressDisplays the current task progress.
Overall ProgressDisplays the overall progress of the File Import.
Import LogText box used to display the File import logs.
Email Logs ToList of email addresses to email a copy of the import log at the end of the sync. Use for multiple list for example t@blogg.com;Accounts@blogg.com.





Import #1 #2 #3


UI ElementDescription
ManagementStudio ModuleSpecify the required module to update.
Create New Module ItemsCreate a new item if no matching record is found in eisting items.
Update Existing Module ItemsUpdate an existing item if a matching record is found in the imported data.
Post-Process Data ScriptIdProcess the imported data through this script before sending it for further processing.
Pre-Process Data ScriptIdScript to run before reading the import file, this can be used to put the import file in place.
File/Folder/UrlA File, Folder or Url of the Files (s) to import. This location must be accessible from the ManagementStudio Server.
File FilterIf a folder path was provided a filter on which files to import can be applied. E.g. 'My_Device_import_*.csv
Excel Worksheet NamesA comma delimited list of worksheet names to import if the source is an Excel document.
Adv. OptionsUse the toggle to hide or show the advanced options.
Move to ProcessMove a matched item to this process using a specified path (e.g. 1. Discovery\Import).
Add to Blueprint 1Add a new or existing Item to this Blueprint using a specified path (e.g. Imports\ServiceNow).
Add to Blueprint 2Add a new or existing Item to this Blueprint using a specified path (e.g. Imports\ServiceNow).
Import BlanksForce imported blank values to overwrite existing data (default behaviour prevents overwriting with blank values).
Import Multiple Matched RowsImport more than one matched row for an item. Useful for importing Blueprints, Contacts, Dependencies etc.
Field MappingCreate a mapping table to link source fields to ManagementStudio (MS) fields. Use the 'M' field to identify matching fields. Example Match App Vendor, Name, and Version from source data to MS data
Find/ReplaceApply basic pre-processing to the source data before importing. Use this option for find and replace operations. Example replace 'Adobe Inc' with 'Adobe'.
Test RetrievalRun the data retrieval portion of the connector to preview the results. It doesn't import the data.
Test ImportRun the data retrieval and import portion of the connector in preview mode. It doesn't import the data.



Field Mapping 

The Field Mapping table describes the relationship between the incoming data and ManagementStudio. 


  • MCH
    • Stands for Match On
    • When this is ticked, ManagementStudio will try to find an existing item that meets this criteria and update that item
    • Example: AppVendor, AppName, AppVersion or SamAccount, Domain
  • Src. Field 
    • The column name from the incoming data 
  • MS Field 
    • The field name in ManagementStudio to insert/update the data. 
    • Supports Name, Custom Field Id, Custom FieldPath, Special fields
    • Exmaples
      • AppName, AppVenor, AppVersion etc
      • 12324  (Saves data to Custom Field Id 1234)
      • APPDSC\General\My Text (Saves data to Custom Field Path)
      • BlueprintId / BlueprintPath
      • AppStatus (Accepted, Pending, Rejected, Rationalised)

  • Options
    • Additional options that can be used change how data is imported
    • Static
      • The Static option takes the literal value in the Src. Field and updates in the MS Field but does not read this value from the import data. 
      • A typical use case
        • Setting a Process e.g.  Src : '2. Imported' MS: 'Process' Opt: 'Static'
        • Setting a import date e.g  Src : '[Today]' MS: 'CustomDate1' Opt: 'Static'




Find / Replace

The imported data can be post-processed before being handed over to ManagementStudio for import. The Find / Replace table allows you to perform some light data cleaning and supports some built-in functions. 


  • Src. Field
    • The column name from the incoming data
  • Find
    • The value to find in the data
  • Replace
    • The value to replace found matches with
  • Func
    • Built-in functions to perform advanced operations on the imported data
    • Regex-FullReplace
      • Look for any occurrence of the Find and replace the whole import value with the Replace
      • e.g. Find 'AMD' replace 'AMD'  :: 'AMD Ryzen 3 7330U' -> 'AMD'
    • Regex-PartialReplace
      • Look for any occurrence of the Find and replace the specific found value with the Replace
      • e.g. Find '7330U ' replace ''  ::  'AMD Ryzen 3 7330U' -> 'AMD Ryzen 3'
    • Convert-BitPackedNumberToVersion
      • Takes a bit-packed number usually found in Parquet files and transforms it into a version 
      • e.g. Convert '6755446685696070' -> 20.3.1.4
    • Convert-Int96DateToDateTime
      • Takes an Int96 formatted Date found in Parquet files and transforms it into a standard date
      • e.g. Convert 'D34A940000001C0E00258CD5' -> 2025-08-07 22:31:11



Post Processor Script

A custom script that can be run on the data retrieved from the File, SQL, or REST API source, but before it is passed through to the importer. 



Pre / Post Processor Script

$ScriptArgs.EventArg1: Name of the import block that called this code. 

e.g. Import1-PreProcess, Import1-PostProcess, Import2-PreProcess, Import2-PostProcess etc

$blockName = $ScriptArgs.EventArg1 


$ScriptArgs.EventArg2: A comma-delimited list of files to process

$files = $ScriptArgs.EventArg2.Split(",",[System.StringSplitOptions]::RemoveEmptyEntries)


Returning an Error will end the current import block, but 

New-MSApiResult -Header "No files found to process" -Status Error


Returning an Exception will end the current import block and end all subsequent import blocks

New-MSApiResult -Header "No files found to process" -Status Exception


Example Script


## Importer Processing Scripts

# EventArg1: Name of the import block that called this code. 
# e.g. Import1-PreProcess
$blockName = $ScriptArgs.EventArg1 

# EventArg2: A comma-delimited list of files  to process
$files = $ScriptArgs.EventArg2.Split(",",[System.StringSplitOptions]::RemoveEmptyEntries)


## Example 1. Exit importer if there are no files to import
if($files.count -eq 0)
{
    New-MSApiResult -Header "No files found to process" -Status Exception
    return
}

## Example 2. Skip import if the number of rows to import < 3
foreach($file in $files)
{
    $lineCount = (Get-Content $file | Measure-Object -Line).Lines
    if($lineCount -lt 3){
        New-MSApiResult -Header "Bad import file, not enough data rows" -Status Error
    }
}






Further Support

If you require further support, please visit ManagementStudio's Service Desk to search the knowledge base or create a new support ticket.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article