Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/letalys/convertfrom-robocoplog
A Powershell Robocopy Log Parser.
https://github.com/letalys/convertfrom-robocoplog
powershell powershell-script powershell-scripts robocopy windows
Last synced: 10 days ago
JSON representation
A Powershell Robocopy Log Parser.
- Host: GitHub
- URL: https://github.com/letalys/convertfrom-robocoplog
- Owner: Letalys
- License: mit
- Created: 2024-10-22T21:13:13.000Z (15 days ago)
- Default Branch: main
- Last Pushed: 2024-10-26T21:50:12.000Z (11 days ago)
- Last Synced: 2024-10-26T23:06:18.298Z (11 days ago)
- Topics: powershell, powershell-script, powershell-scripts, robocopy, windows
- Language: PowerShell
- Homepage:
- Size: 98.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ConvertFrom-RobocopLog : Universal Robocopy Log Parser
`ConvertFrom-RobocopLog.ps1` is a powerful PowerShell tool designed to analyze log files generated by Robocopy, providing a clear and structured overview of file copy operations.
It converts logs into information-rich PowerShell objects, facilitating automation, reporting, and analysis of data transfers.
This script support various Robocopy log formats with different log options, including those with full or partial file paths.
- Detailed extraction of header information (source, destination, options).
- Complete analysis of copied, modified, error or ignored files.
- Processing of overall statistics (number of files, bytes, errors) with the possibility of calculating the total processing time.
- `ParseNoFiles` option to quickly retrieve a preview of the first and last sections of the log, speeding up analysis when file details are not needed.
This script is ideal for system administrators looking to better understand or automate the management of Robocopy operations in complex environments, while maintaining an accurate view of performance and results.
## Prerequisites
To use ConvertFrom-RobocopLog.ps1, you must ensure that your environment meets the following prerequisites:
- **Operating system:** Windows with PowerShell (version 5.1 or later).
- **Robocopy:** The log file must be generated by a Robocopy command with the log generation option enabled (eg: /LOG, /UNILOG).
- **Permissions:** You must have the necessary permissions to read the specified log file.
- **Full file paths:** For analysis, Robocopy should be run with the `/fp` (Full Path) option to log full file paths.
- **Show Class:** For analysis, Robocopy should be run **without** the `/nc` (Class not logged) option to log full file paths.
For French Log, use `/unilog`option instead of `/log`
## How to use RobocopLog
### First Way : Use `ConvertFrom-RobocopLog.ps1` file directly
To use ConvertFrom-RobocopLog.ps1, specify the Robocopy log file to analyze using the required -RoboLog parameter. The script will parse the content and return a structured PowerShell object containing the header information, processed files, and summary statistics.
- `-RoboLog` : Path to the Robocopy log file to analyze (required).
- `-LogLanguage` : Log language (optional, possible values: "en-US" or "fr-FR", default "en-US").
- `-ParseType` : File parsing method.
- `"Full"` : Complete analysis of the log file (default).
- `"NoParseFile"` : Parses only the first 15 and last 15 lines of the log file for a quick overview.
- Use this option in the case of a large log to only have the start and end report because log processing can be very long.
### Second Way : Get the internal function and integrate it into your own scripts
You can copy/paste the function `ConvertFrom-RobocopLog { ...}` and either declare it in your script, but you can also create your own .psm1 module to integrate it and not pollute your script.
The settings are completely identical.
## Examples of use
Complete analysis of a log file in English (default):
```
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt" -ParseType Full -LogLanguage en-US
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt"
```
Analysis of a log file in French with quick parsing (without file details):
```
$LOG = .\ConvertFrom-RobocopLog.ps1 -RoboLog "C:\Logs\RobocopyLog.txt" -LogLanguage fr-FR -ParseType NoParseFile
```
### Object Properties
1. `HeaderInfo`: Contains log header information.
- `Start` : Start date and time.
- `Source` : Transfer source.
- `Destination` : Destination of the transfer.
- `Files` : File criteria.
- `Options` : Options used with Robocopy
**Exemple**
```
$LOG.HeaderInfo.Start # 2024/09/25 08:14:10
$LOG.HeaderInfo.Source # C:\SourceFolder\
$LOG.HeaderInfo | Format-List
######
Start : 2024/09/25 08:14:10
Source : C:\SourceFolder\
Destination : C:\DestFolder\
FilesOptions : *.*
Options : *.* /V /X /TS /FP /S /E /COPYALL /PURGE /MIR /NP /R:1000000 /W:30
```
2. `Files`: Contains information about processed files according to their class
- `New`,`Same`,`Newer`,`Older`,`Modified`,`Lonely`,`Tweaked`
- `FilePath` : Full path of file
- `FileSize` : With `/byte` or with default Robocopy unit (m, g, t, ..)
- `TimeStamp`
- `Failed` : Contains the list of files with error.
- `FilePath` : Full path of file
- `ErrorInfo` : Code error
- `ErroAction` : Indicates what action was attempted (Copy, Delete, etc.)
- `Destination` : Destination of the transfer.
- `Files` : File criteria.
- `Options` : Options used with Robocopy
**Exemple**
```
$LOG.Files.Extra
######
FilePath FileSize Timestamp
-------- -------- ---------
C:\SourceFolder\LargeFile_1.dat 1.0 g 25/09/2024 14:11:56
C:\SourceFolder\LargeFile_2.dat 1.0 g 25/09/2024 14:12:00
$LOG.Files.New | Out-GridView
$LOG.Files.Failed
######
FilePath ErrorInfo ErrorAction
-------- --------- -----------
C:\SourceFolder\LargeFile_10.dat ERROR 32 0x00000020 Copying File
C:\SourceFolder\LargeFile_11.dat ERROR 32 0x00000020 Copying File
C:\SourceFolder\LargeFile_12.dat ERROR 32 0x00000020 Copying File
```
3. `Dirs`: Contains information about processed Directory according to their class
- `New`, `Extra`, `Lonely`
- `Dirpath`
- `ItemCount` : Count Inner Directory Object (-1 if Deleting)
**Exemple**
```
$log.Dirs.New[0].DirPath # "C:\SourceFolder\NewDirectory\"
```
4. `SummaryInfo`: Overall summary of operations.
- `Dirs` :
- `Total`, `Copied`, `Skipped`, `Mismatch`, `Failed`, `Extras`.
- `Files` :
- `Total`, `Copied`, `Skipped`, `Mismatch`, `Failed, `Extras`.
- `Ended` : End date and time.
- `TotalTimeFromInnerLog` : Total duration calculated from Header et Summary Data Date.
- `TotalTimeFromFile` : Total duration calculated from log file metadata
**Exemple**
```
$LOG.SummaryInfo
######
Dirs : @{Total=2; Copied=1; Skipped=1; Mismatch=0; Failed=0; Extras=1}
Files : @{Total=18; Copied=3; Skipped=15; Mismatch=0; Failed=0; Extras=2}
Ended : 2024/09/25 08:25:10
TotalTimeFromInnerLog : 00d:00h:00m:00s
TotalTimeFromFile : 00d:00h:53m:45s
$LOG.SummaryInfo.Files.Skipped # 15
```
## Supported Robocopy Log language, Options and FileClass
### Language
|Language | Comments |
|---------|:--------------------------------------------------------------|
|English | By Default, can use with `/log` option from robocopy command. |
|French | By Default, can use with `/unilog` option from robocopy command. Needed to handle accents |
### Add Your Language KeyWord
Just add Your Language CultureMap From the variable `$CultureMap = @{...}`.
Then remember to add your language option to the function and/or script parameter.
```
[Parameter(Mandatory = $false)]
[ValidateSet("en-US", "fr-FR")]
[String]$LogLanguage = "en-US",
```
### Options (Non-exhaustive list)
**This section covers the options that Robocopy must use to generate its log in order to be analyzed by RobocopLog**
*For more information about Robocopy options click on the link: [Microsoft Learn : Robocopy](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy)*
**JOB Options** : Not tested.
| Options | Supported | Microsoft Description |
|:------------:|:-----------:|:--------------------------------------------------------------|
|/xf | no | Excludes files that match the specified names or paths. Wildcard characters (* and ?) are supporte |
|/nc | no | Excludes directories that match the specified names and paths. |
|Log Options | Supported | Comments | Microsoft Description |
|:------------:|:-----------:|:----------------------------------------------------------------------------------------|:--------------------------------------------------------------|
|/fp | yes | **Mandatory** : FullPath need to be generated for bein captured by parsing |Includes the full path names of the files in the output. |
|/nc | no | **Forbidden** : Class need to be generated during log because they are used for parsing |Specifies that file classes aren't to be logged. |
|Log Options | Supported | Recommended* | Microsoft Description |
|:------------:|:-----------:|:------------:|:----------------------|
|/log+ | **no** | - |Writes the status output to the log file (appends the output to the existing log file).|
|/unilog+ | **no** | - |Writes the status output to the log file (appends the output to the existing log file).|
|/unicode | **no** | - |Displays the status output as unicode text. |
|/v | yes | yes |Produces verbose output, and shows all skipped files.|
|/ts | yes | yes |Includes source file time stamps in the output.|
|/byte | yes | yes |Prints sizes as bytes.|
|/ns | yes | no |Specifies that file sizes aren't to be logged.|
|/np | yes | yes |Specifies that the progress of the copying operation (the number of files or directories copied so far) won't be displayed. |
|/ndl | yes | no |Specifies that directory names aren't to be logged.|
|/nfl | yes | no |Specifies that file names aren't to be logged.|
|/njh | yes | no |Specifies that there's no job header.|
|/njs | yes | no |Specifies that there's no job summary.|
|/x | yes | yes |Reports all extra files, not just the ones that are selected.|
\* ***Note** : Recommended for Optimal Analysis*
### Robocopy Files Class Information
#### English
```
File Exists In Exists In Source/Dest Source/Dest Source/Dest
Class Source Destination File Times File Sizes Attributes
=========== =========== ================ =============== ============= ============
Lonely Yes No n/a n/a n/a
Tweaked Yes Yes Equal Equal Different
Same Yes Yes Equal Equal Equal
Changed Yes Yes Equal Different n/a
Newer Yes Yes Source > Dest n/a n/a
Older Yes Yes Source < Dest n/a n/a
Extra No Yes n/a n/a n/a
Mismatched Yes (file) Yes (directory) n/a n/a n/a
```
#### French
```
Classe de Existe dans Existe dans Source/Dest Source/Dest Source/Dest
Fichier la Source la Destination Date fichier Taille fichier Attributs
=========== =========== ================ =============== ============= ============
Solitaire Oui Non n/a n/a n/a
Tweaked * Oui Oui Identique Identique Différent
Identique Oui Oui Equal Identique Identique
Modifié Oui Oui Equal Different n/a
Plus récent Oui Oui Source > Dest n/a n/a
Plus ancien Oui Oui Source < Dest n/a n/a
Supplément. Non Oui n/a n/a n/a
Discordance Yes (Fic) Yes (Rép) n/a n/a n/a
```
\* ***Note** : Tweaked is KeyWord used for french too.*
## Releases
- Version `1.0` | 26/09/2024
- Version `1.01` | 27/09/2024
- Bug fix : When the $ParseType option was full, the end summary was not correctly parsed due to a bad condition
## Links
- [https://github.com/Letalys/Powershell-ConvertFrom-RobocopLog](https://github.com/Letalys/Powershell-ConvertFrom-RobocopLog)
## Autor
- [@Letalys (GitHUb)](https://www.github.com/Letalys)