Download and extract archive using PowerShell

by Magnar Myrtveit   Last Updated May 23, 2020 01:01 AM

Using PowerShell 5.1, how can I download a tar.xz archive and extract it without writing it to disk first?

All these attempts:

Invoke-WebRequest https://www.examle.com/archive.tar.xz -UseBasicParsing | 7z x -si
(Invoke-WebRequest https://www.examle.com/archive.tar.xz -UseBasicParsing).ToString() | 7z x -si
(Invoke-WebRequest https://www.examle.com/archive.tar.xz -UseBasicParsing).Content | 7z x -si
(Invoke-WebRequest https://www.examle.com/archive.tar.xz -UseBasicParsing).RawContent | 7z x -si

Gives this error:

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Extracting archive:
ERROR:
Can not open encrypted archive. Wrong password?

Not implemented

Can't open as archive: 1
Files: 0
Size:       0
Compressed: 0

This works:

Invoke-WebRequest https://www.examle.com/archive.tar.xz -UseBasicParsing -OutFile temp.tar.xz
7z x temp.tar.xz


Answers 1


Challenge: Download, extract xz archive in PowerShell

Without writing anything to disk

Just so we have a real example, suppose we want to download, extract the mingw32-dev.tar.xz XZ archive from MinGW - Minimalist GNU for Windows.

We can download without writing to disk as follows:

$r=Invoke-WebRequest -Uri 'https://mirrors.gigenet.com/OSDN//mingw/70554/mingwrt-5.2.1-mingw32-dev.tar.xz'

So now our target archive is available to shell as a byte array in r.Content. How to extract?

This is possible using SevenZipExtractor C# wrapper for 7Zip.

#Download and install from nuget.org (needs admin privileges)
Install-Package SevenZipExtractor

#Add the SevenZip assembly to our current PowerShell session
(Get-Item (Join-Path (Split-Path (Get-Package SevenZipExtractor).Source) lib/netstandard*) | 
  Sort-Object { [version] ($_.Name -replace '^netstandard') })[-1] |
    Get-ChildItem -Filter *.dll -Recurse |
      ForEach-Object { Write-Host "Adding ``$($_.Name)``"; Add-Type -LiteralPath $_.FullName }

The SevenZipExtractor class includes the following overloaded constructor signatures (it also has others we are ignoring):

public SevenZipExtractor(Stream archiveStream);
public SevenZipExtractor(Stream archiveStream, string password);
public SevenZipExtractor(Stream archiveStream, InArchiveFormat format);
public SevenZipExtractor(Stream archiveStream, string password, InArchiveFormat format);

where Stream means type System.IO.Stream and InArchiveFormat means type SevenZip.InArchiveFormat. So we can use the SevenZipExtractor class by

$sevenZipStream = [System.IO.MemoryStream]::new()
$sevenZipStream.Write($r.Content,0,$r.Content.Length)
New-Object -TypeName SevenZipExtractor -ArgumentList @($sevenZipStream,[SevenZip.InArchiveFormat]::XZ)

Writing to temp file to disk

If we were writing the file to disk, this would be more simple to achieve using the 7Zip4Powershell module.

Install-Module -Name 7Zip4Powershell
Import-Module -Name 7Zip4Powershell -Global

However, 7Zip4PowerShell does not implement all the overloaded method signatures of SevenZipExtractor Since the Windows 10 Preview Build 17063, bsdtar is included with PowerShell. To extract a tar.xz file, invoke the tar command with the --extract (-x) option and specify the archive file name after the -f option:

tar -xf .\whatever.xz

tar auto-detects the compression type and extracts the archive. For more verbose output, use the -v option. This option tells tar to display the names of the files being extracted on the terminal.

Ordinary zip file in memory

Here is an example in which I find aes.h inside the ffmpeg source code zip archive:

    [System.Reflection.Assembly]::LoadWithPartialName('System.IO.Compression')
    $IWRresult = Invoke-WebRequest -Uri "https://ffmpeg.zeranoe.com/builds/win64/dev/ffmpeg-latest-win64-dev.zip"  -SslProtocol Tls12 -Method Get
    $zipStream = New-Object System.IO.Memorystream
    $zipStream.Write($IWRresult.Content,0,$IWRresult.Content.Length)
    $zipFile = [System.IO.Compression.ZipArchive]::new($zipStream)
    #OK, what's in the archive I just downloaded?
    #Write the archive contents to the shell output
    $zipFile.Entries | Select-Object -ExcludeProperty @('Archive','ExternalAttributes') | Format-Table #I don't care about 'Archive' or 'ExternalAttributes', so I instruct suppress those
    #oh, there's my `aes.h` inside `ffmpeg-latest-win64-dev/include/libavutil/`
    $entry =  $zipFile.GetEntry('ffmpeg-latest-win64-dev/include/libavutil/aes.h')
    #now we have a streamreader, we can do all the things
    #for example, let's output the content to the screen
    $reader = [System.IO.StreamReader]::new($entry.Open())
    Write-Host $reader.ReadToEnd()

Another example that aims to download archive, run EXE from that archive all in memory:

    [System.Reflection.Assembly]::LoadWithPartialName('System.IO.Compression')
    $IWRresult=Invoke-WebRequest -Uri 'https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20200424-a501947-win64-static.zip' -Method Get -SslProtocol Tls12
    $zipStream = New-Object System.IO.Memorystream
    $zipStream.Write($IWRresult.Content,0,$IWRresult.Content.Length)
    $zipFile = [System.IO.Compression.ZipArchive]::new($zipStream)
    #OK, what did I just download?
    #Write the contents to the shell output
    $zipFile.Entries | Select-Object -ExcludeProperty @('Archive','ExternalAttributes') | Format-Table #I don't care about 'Archive' or 'ExternalAttributes', so I instruct suppress those
    #I see there is 'ffmpeg-20200424-a501947-win64-static/bin/ffmpeg.exe' entry
    $zipEntry = $zipFile.GetEntry('ffmpeg-20200424-a501947-win64-static/bin/ffmpeg.exe')
    $binReader = [System.IO.BinaryReader]::new($ffmpeg.Open())
    #need external modules `PowerShellMafia/PowerSploit` to be able to run exe from memory (without writing to disk); see comments below this code block
    Invoke-ReflectivePEInjection -PEBytes $binReader.ReadBytes() -ExeArgs "Arg1 Arg2 Arg3 Arg4"

Please see PowerShellMafia/PowerSploit for more information about how to run an EXE from the memory. Find examples here: GitHub Examples

Mavaddat Javid
Mavaddat Javid
April 26, 2020 23:31 PM

Related Questions




Why can't cURL and wget connect to footpatrol.com

Updated July 06, 2019 17:01 PM

Get (only) content on URL on stdout

Updated September 29, 2019 18:01 PM