[Buildroot] [PATCH v2 05/12] support/download/go-post-process: implement Go vendoring support

[Yann E. MORIN]

Christian, All,

On 2021-08-01 00:24 -0700, Christian Stewart spake thusly:
> On Sun, Aug 1, 2021 at 12:08 AM Yann E. MORIN <yann.morin.1998 at free.fr> wrote:
> > So I also hashed the archives you provided, and they too are identical:
> >     $ sha1sum *.tar.gz
> >     bd887a60c4ca60e55d062067132ccab7d85e7d95  embiggen-disk-bad.tar.gz
> >     bd887a60c4ca60e55d062067132ccab7d85e7d95  embiggen-disk-good.tar.gz
> > Did you mess up when sending the archives?
> Not sure how that got mixed up, attached is the correct
> embiggen-disk-bad.tar.gz:

So, what are the differences?

First, looking at the compressed archive is useless. We really need to
look at the uncompressed archives to see what is going on.

    $ gunzip embiggen-disk-good.tar.gz embiggen-disk-bad.tar.gz

    $ hxedump -Cv embiggen-disk-good.tar >embiggen-disk-good.tar.hex
    $ hxedump -Cv embiggen-disk-bad.tar >embiggen-disk-bad.tar.hex

    $ diff -du embiggen-disk-good.tar.hex embiggen-disk-bad.tar.hex >embiggen-disk.diff

    $ diffstat <embiggen-disk.diff
     embiggen-disk-bad.tar.hex |   52    26    26     0 +++++++++++++-------------
     1 file changed, 26 insertions(+), 26 deletions(-)

So, there aren't so many differences, in the end. What's about them,
then? Let's see the first hunk (the others are very similar):

    @@ -4,10 +4,10 @@
     00000030  31 61 32 33 30 38 2f 43  4f 4e 54 52 49 42 55 54 |1a2308/CONTRIBUT|
     00000040  49 4e 47 2e 6d 64 00 00  00 00 00 00 00 00 00 00 |ING.md..........|
     00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
    -00000060  00 00 00 00 30 30 30 30  36 34 34 00 30 30 30 30 |....0000644.0000|
    +00000060  00 00 00 00 30 30 30 30  36 36 34 00 30 30 30 30 |....0000664.0000|
     00000070  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30 |000.0000000.0000|
     00000080  30 30 30 31 33 34 37 00  31 33 36 31 36 36 36 32 |0001347.13616662|
    -00000090  36 31 33 00 30 32 30 34  30 33 00 20 30 00 00 00 |613.020403. 0...|
    +00000090  36 31 33 00 30 32 30 34  30 35 00 20 30 00 00 00 |613.020405. 0...|
     000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
     000000b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
     000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|

So, the first obvious delta: the access mode. In the 'good' archive, it
is 0644 (rw-r--r--), while in the 'bad' archive, it is 0664 (rw-rw-r--).

But Buildroot is supposed to enforce the mode, see Makefile:

    70: REQ_UMASK = 0022
    ...
    77: ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
    ...
    83: _all:
    84:     @umask $(REQ_UMASK) && \
    85:         $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
    86:             $(MAKECMDGOALS) $(EXTRAMAKEARGS)

So, how comes that the two have different modes?

The second change is a bit more curious. As per the GNU tar manual [0]:
offset 0x94 (148) is 'char chksum[8]'.

So we should expect an 8-byte field (still from [0]):

    The chksum field represents the simple sum of all bytes in the
    header block. Each 8-bit byte in the header is added to an unsigned
    integer, initialized to zero [...]

OK, so because the mode is different, the delta from the mode is carried
over to the chksum.

If we can find why the modes are different in your two cases, we can
probably solve the issue.

QED.

Still, I would prefer that we do use the tarball helper, because then we
would have a single location where we could solve this kind of
problems...

[0] https://www.gnu.org/software/tar/manual/html_node/Standard.html

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'