Отправляю запрос на сайт .
Запрос копировал из Google Network полностью, то есть ошибок в нём нет. Вывод контента страницы появляется в некорректной кодировке.
Вывод выполняется в PowerShell ISE
Обезличенный скрипт – запрос
#[Console]::outputEncoding = [System.Text.Encoding]::GetEncoding('cp866')
#[Console]::outputEncoding = [System.Text.Encoding]::GetEncoding(‘windows-1251’)
#[console]::OutputEncoding = [System.Text.Encoding]::UTF8
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$session.UserAgent = "Example"
$session.Cookies.Add((New-Object System.Net.Cookie("Example")))
$wget = Invoke-WebRequest -UseBasicParsing -Uri "https://www.abuseipdb.com/check?query=50.31.21.4" `
-WebSession $session `
-Headers @{
"authority"="www.abuseipdb.com" "method"="GET" "path"="/check?query=50.31.21.4" "scheme"="https" "accept"="text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "accept-encoding"="gzip, deflate, br, zstd" "accept-language"="ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7" "referer"="https://www.abuseipdb.com/" "sec-ch-ua"="`"Chromium`";v=`"122`", `"Not(A:Brand`";v=`"24`", `"Google Chrome`";v=`"122`"" "sec-ch-ua-arch"="`"x86`"" "sec-ch-ua-bitness"="`"64`"" "sec-ch-ua-full-version"="`"122.0.6261.129`"" "sec-ch-ua-full-version-list"="`"Chromium`";v=`"122.0.6261.129`", `"Not(A:Brand`";v=`"24.0.0.0`", `"Google Chrome`";v=`"122.0.6261.129`"" "sec-ch-ua-mobile"="?0" "sec-ch-ua-model"="`"`"" "sec-ch-ua-platform"="`"Windows`"" "sec-ch-ua-platform-version"="`"10.0.0`"" "sec-fetch-dest"="document" "sec-fetch-mode"="navigate" "sec-fetch-site"="same-origin" "sec-fetch-user"="?1" "upgrade-insecure-requests"="1"
}
$wget.ContentПытался изменить кодировку с помощью этих команд по очереди, но ни одна не помогла:
[Console]::outputEncoding = [System.Text.Encoding]::GetEncoding('cp866')
[Console]::outputEncoding = [System.Text.Encoding]::GetEncoding(‘windows-1251’)
[console]::OutputEncoding = [System.Text.Encoding]::UTF8Также менял accept-language на английский, следующим образом:
"accept-language"="ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7"
на "accept-language"="en-UK,en;q=0.9,en-US;q=0.8,en;q=0.7" Но не помогло.
Подскажите пожалуйста, как я могу сменить кодировку на читаемую?
Заранее благодарю за ответ!
In addition, on GitHub, the SQL files we committed to the GitHub are all marked as binary files. Thus we couldn’t view the changes we made to those files in the commit.
Cause of the Issue
It turns out that those SQL files are generated from SQL Server Management Studio (SSMS).
By default, the encoding used to save SQL files in SSMS is UTF-16. For my case, my default encoding is the “Western European (Windows) – Codepage 1252”. Codepage 1252 is a single-byte character encoding of the Latin alphabet that was used in Windows for English and many Romance and Germanic languages. This encoding will cause Git to treat the files as binary files.
Solution
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
Get-ChildItem "<absolute directory path>" -Recurse *.sql | foreach { $FilePath = $_.FullName $FileContent = Get-Content $FilePath [System.IO.File]::WriteAllLines($FilePath, $FileContent, $Utf8NoBomEncoding)
}The BOM (Byte Order Mark), a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF), is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is unnecessary. This explains why we pass $False to the constructor of UTF8Encoding to indicate that BOM is not needed.
Wrap-Up
That’s all for a short little PowerShell script we used to solve the encoding issue of our SQL files.
There is an interesting discussion on StackOverflow about this issue, please give it a read too.
KOSD, or Kopi-O Siew Dai, is a type of Singapore coffee that I enjoy. It is basically a cup of coffee with a little bit of sugar. This series is meant to blog about technical knowledge that I gained while having a small cup of Kopi-O Siew Dai.
