[Buildroot] Github download helper possibly not working

[Thomas Petazzoni]

On Mon, 28 Aug 2023 20:02:34 +0000
Woody Douglass via buildroot <buildroot at buildroot.org> wrote:

> I've noticed that packages that use the `github` download helper are 
> falling over to buildroot mirrors. I've tried with packages `yaml-cpp`, 
> `zlog`, `swupdate`, and `pcm-tools` -- all are redirected before 
> eventually getting a 403 from codeload.github.com and falling back to 
> sources.buildroot.net. Is anyone else seeing this behavior? I'm trying 
> to find a workaround now, but I'd appreciate any help!

It works fine here:

$ make yaml-cpp-source
>>> yaml-cpp 0.7.0 Downloading
wget --passive-ftp -nd -t 3 -O '/home/thomas/projets/buildroot/output/build/.yaml-cpp-0.7.0.tar.gz.TezLTj/output' 'https://github.com/jbeder/yaml-cpp/archive/yaml-cpp-0.7.0/yaml-cpp-0.7.0.tar.gz' 
--2023-08-28 22:43:34--  https://github.com/jbeder/yaml-cpp/archive/yaml-cpp-0.7.0/yaml-cpp-0.7.0.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/jbeder/yaml-cpp/tar.gz/refs/tags/yaml-cpp-0.7.0 [following]
--2023-08-28 22:43:34--  https://codeload.github.com/jbeder/yaml-cpp/tar.gz/refs/tags/yaml-cpp-0.7.0
Resolving codeload.github.com (codeload.github.com)... 140.82.121.10
Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1033237 (1009K) [application/x-gzip]
Saving to: ‘/home/thomas/projets/buildroot/output/build/.yaml-cpp-0.7.0.tar.gz.TezLTj/output’

/home/thomas/projets/buildroot/ 100%[=====================================================>]   1009K   923KB/s    in 1,1s    

2023-08-28 22:43:36 (923 KB/s) - ‘/home/thomas/projets/buildroot/output/build/.yaml-cpp-0.7.0.tar.gz.TezLTj/output’ saved [1033237/1033237]

yaml-cpp-0.7.0.tar.gz: OK (sha256: 43e6a9fcb146ad871515f0d0873947e5d497a1c9c60c58cb102a97b47208b7c3)

$ make zlog-source
>>> zlog 1.2.16 Downloading
wget --passive-ftp -nd -t 3 -O '/home/thomas/projets/buildroot/output/build/.zlog-1.2.16.tar.gz.jktoPe/output' 'https://github.com/HardySimpson/zlog/archive/1.2.16/zlog-1.2.16.tar.gz' 
--2023-08-28 22:43:51--  https://github.com/HardySimpson/zlog/archive/1.2.16/zlog-1.2.16.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/HardySimpson/zlog/tar.gz/refs/tags/1.2.16 [following]
--2023-08-28 22:43:51--  https://codeload.github.com/HardySimpson/zlog/tar.gz/refs/tags/1.2.16
Resolving codeload.github.com (codeload.github.com)... 140.82.121.10
Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘/home/thomas/projets/buildroot/output/build/.zlog-1.2.16.tar.gz.jktoPe/output’

/home/thomas/projets/buildroot/     [ <=>                                                  ] 122,20K  --.-KB/s    in 0,1s    

2023-08-28 22:43:52 (1,08 MB/s) - ‘/home/thomas/projets/buildroot/output/build/.zlog-1.2.16.tar.gz.jktoPe/output’ saved [125131]

zlog-1.2.16.tar.gz: OK (sha256: 742401902f2134eb272c49631fe5c38d7aeb9a2ad56fa3ec3d15219b371ba655)

$ make swupdate-source
>>> swupdate 2022.12 Downloading
wget --passive-ftp -nd -t 3 -O '/home/thomas/projets/buildroot/output/build/.swupdate-2022.12.tar.gz.ofwOCw/output' 'https://github.com/sbabic/swupdate/archive/2022.12/swupdate-2022.12.tar.gz' 
--2023-08-28 22:44:17--  https://github.com/sbabic/swupdate/archive/2022.12/swupdate-2022.12.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/sbabic/swupdate/tar.gz/refs/tags/2022.12 [following]
--2023-08-28 22:44:17--  https://codeload.github.com/sbabic/swupdate/tar.gz/refs/tags/2022.12
Resolving codeload.github.com (codeload.github.com)... 140.82.121.10
Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘/home/thomas/projets/buildroot/output/build/.swupdate-2022.12.tar.gz.ofwOCw/output’

/home/thomas/projets/buildroot/     [              <=>                                     ]   6,63M  2,42MB/s    in 2,7s    

2023-08-28 22:44:20 (2,42 MB/s) - ‘/home/thomas/projets/buildroot/output/build/.swupdate-2022.12.tar.gz.ofwOCw/output’ saved [6957587]

swupdate-2022.12.tar.gz: OK (sha256: e6335e87812a98a87f1c55df03c9f4e4ef042789570002c5db120b09f64b0d86)

However, for pcm-tools, we do have a problem (though not the one you
mentioned):

$ make pcm-tools-source
>>> pcm-tools 202110 Downloading
wget --passive-ftp -nd -t 3 -O '/home/thomas/projets/buildroot/output/build/.pcm-tools-202110.tar.gz.39EDUL/output' 'https://github.com/opcm/pcm/archive/202110/pcm-tools-202110.tar.gz' 
--2023-08-28 22:44:37--  https://github.com/opcm/pcm/archive/202110/pcm-tools-202110.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/intel/pcm/archive/202110/pcm-tools-202110.tar.gz [following]
--2023-08-28 22:44:38--  https://github.com/intel/pcm/archive/202110/pcm-tools-202110.tar.gz
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/intel/pcm/tar.gz/refs/tags/202110 [following]
--2023-08-28 22:44:38--  https://codeload.github.com/intel/pcm/tar.gz/refs/tags/202110
Resolving codeload.github.com (codeload.github.com)... 140.82.121.9
Connecting to codeload.github.com (codeload.github.com)|140.82.121.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘/home/thomas/projets/buildroot/output/build/.pcm-tools-202110.tar.gz.39EDUL/output’

/home/thomas/projets/buildroot/     [   <=>                                                ]   1,15M  2,18MB/s    in 0,5s    

2023-08-28 22:44:39 (2,18 MB/s) - ‘/home/thomas/projets/buildroot/output/build/.pcm-tools-202110.tar.gz.39EDUL/output’ saved [1205829]

ERROR: pcm-tools-202110.tar.gz has wrong sha256 hash:
ERROR: expected: aa48ab1473720aeb7837b67bfc612100f484748720a8b8034daff00419709057
ERROR: got     : 90a5931cea24f1b0da76e22c712e55375df157e87f26edaa70b9660405852725
ERROR: Incomplete download, or man-in-the-middle (MITM) attack

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com