Here is my scenario
Invoke-sqlcmd -query "Select * from master.sys.table" -ServerInstance myserver -database -mydb | convert-csv | select -skip 2 | .\7z.exe a -si tgzip $gzipedfilename
Now I am taking that $gzipedfilename and uploading to blob storage using azcopy.
This is causing huge disk I/O and queue when I am running above script in multiple threads. I am trying pipe Azcopy on the output of above script by doing
| .\7z.exe a -si tgzip -so | Azcopy -----
but I don't know if Azcopy supports piping and how. Can someone help? Or s there any other way to accomplish this by bypassing entire intermediate zipped file?
Using azcopy_windows_amd64_10.2.1 I checked their help
.\azcopy.exe copy --help
and this shows that you can, at least now with block blobs, use piping
Upload a single file with a SAS using piping (block blobs only):
- cat "/path/to/file.txt" | azcopy cp "https://..../[container]/[path/to/blob]?[SAS]"
Yes, now it supports example:
cat testfile.txt | azcopy copy "https://[storage-account].blob.core.windows.net/[container]/[filename]?[sas_token]" --from-to PipeBlob
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With