1-pixel wide black line in generated +Y cube face image

[emzo]

platform: macOS 11.6
python version: 3.8.5
vips version: 8.12.2
pyvips version: 2.2.1
numpy version: 1.23.1
kubi version: master

First of all, thanks for developing kubi - it's super useful, especially for generating cube map images for use with Marzipano. I keep encountering a problem with all the images I try as inputs - the output +Y image always seemes to have a 1-pixel wide black line down the middle from the top to the center of the image, like in the image below:

I think I've tracked the problem down to the way the x,y coordinates for the mapping image (for use with the vips mapim function) are generated. It seems that some of the pixel coordinates fall outide the bounds of the input image. Given an input image with dimentions of 4096x2049 pixels, the mapping for face +Y tried to access pixels at x coordinates 4096. Since image dimentions are zero-based, the maximum x coordinate would be 4095.

I shifted the linspace sample values by half a step on line 64, and this seems to have fixed the issue, and cube edges seem to line up more precisely now too:

Cubemap in Marzipano without the fix:

Cubemap in Marzipano with the fix applied:

[indus]

I have never noticed that. Are you sure with your input dimensions? 4096x2049 pixels? Shouldn't it be 2048 (power of two)?
I will have a look into that.

[indus]

I'm unable to reproduce the 1px line you are seeing in my setup:

platform: win 10 (yours: macOS 11.6)
python version: 3.7.0 (yours: 3.8.5)
vips version: 8.12.2
pyvips version: 2.2.1
numpy version: 1.21.6 (yours: 1.23.1; I can't install a higher version for some reason)
kubi version: master

I've tested the basemap example starting with the tiff:

kubi --vips D:/User/Downloads/vips-dev-w64-all-8.12.2/vips-dev-8.12/bin -s 512 ./tests/in/basemap.tif ./tests/out/test.png

I did multiple resolutions and could't find an artefact. Have you used the resampling option -r ?

[indus]

I'm still investigating, but can't find a problem.
endpoint=False in the linspace should ensure values from [0:size-1] that fit to the index of the input image without an overflow.

To make a good comparison, could you please create an input like so:

import pyvips
img = pyvips.Image.black(1,256)
img = img.join((img + 255),0)
for x in range(8):
    img = img.join(img,0)
img.write_to_file("./test.png")

and then run kubi like so:

kubi test.png

The result test_cubemap_2.png should look perfectly symmetrical; like this:

Can you upload your result for comparison?

[emzo]

Are you sure with your input dimensions? 4096x2049 pixels? Shouldn't it be 2048 (power of two)?

Yes, apologies, that was a typo! I meant 4096x2048.
The images I've been using are all power of two 2:1 aspect ratio

Have you used the resampling option -r ?

I've tried with and without the resampling option, but the results are the same.

[indus]

I just saw that the latest libvips version (8.13) has a new extend option in mapim: https://www.libvips.org/API/current/libvips-resample.html#vips-mapim

VIPS_EXTEND_COPY could be a solution to the 1px border as well...

[emzo]

Sorry for the delay in responding, but I'm having a hard time figuring out exactly where the root of this problem lies.

This may well be unrelated to the original issue, but I was stumped for a while because the code you provided above to generate a test image seemed to result in a totally black image for me. However, after closer inspection, what it's actually generating is a 16-bit PNG file, so while it does look totally black, it does actually have a striped pattern to it (it's just that the stripes are black and really dark grey rather than black and white).

I imagine the source test image should be an 8-bit PNG that looks like the one below (black-white-black-white... vertical stripes):

On my setup, the generated image looks like this (there are vertical stripes here, it's just that they are very dark since pixel values of 255 in a 16-bit image is a very dark grey):

It's strange that your code above generated an 8-bit image on your setup, and a 16-bit image on mine.

[emzo]

Back to the main issue though... if you don't mind me suggesting, I think a better test image would be the horizontally flipped version of your test image, where the stripes are essentially inverted (white-black-white-black..., rather than black-white-black-white...).

Running kubi on this should produce the inverse of your example pattern above with white running down the centre of the image rather than black (which could be masking the problem?)

Running kubi test.png on this image:

gives me the following output, which is essentially the same as your test but with the colours inverted - you can see the black line is visible in the +Y cube face on my setup:

Would you be so kind as to run kubi against the new (white-black-white-black...) source image above to see what you get?

[emzo]

Can you upload your result for comparison?

With #4 applied, the output image is slightly different (due to the different sampling/mapping points), but is still perfectly symmetrical.

Here's the output (for the +Y cube face) with your original test image (black-white-black-white...) and #4 applied:

And here is the same on the flipped test image (white-black-white-black...) with #4 applied:

[emzo]

I'm unable to reproduce the 1px line you are seeing in my setup:

numpy version: 1.21.6 (yours: 1.23.1; I can't install a higher version for some reason)

I also tried with NumPy 1.21.6 (like you have installed on your setup), but still seeing the black line.

[indus]

Thats all quite strange...
Regarding the "test.png" I get a 8 bit PNG in Ubuntu (WSL) but a 16 bit image in Windows.
Nevertheless your flipped version is way more meaningful for a test. I used it and got the correct result (what I belief to be the correct result):

I don't see a chance to merge your PR right now as long as I don't know what is actually causing these differences in the output.

[emzo]

Thats all quite strange...

Indeed it is - I'm no closer to finding why I'm getting these results 😅

Regarding the "test.png" I get a 8 bit PNG in Ubuntu (WSL) but a 16 bit image in Windows.

That sounds like an issue with pyvips/libvips - it should surely work the same regardless of operating system.

Nevertheless your flipped version is way more meaningful for a test. I used it and got the correct result (what I belief to be the correct result):

Yes, your result an mine look identical except for the black line on mine 🤔

I don't see a chance to merge your PR right now as long as I don't know what is actually causing these differences in the output.

I totally understand. I can continue to use my fork for now, but I'll keep looking into it, and maybe try it on Windows/Linux as well. Thanks for taking the time to look into the issue for me.

[claytonrothschild]

I also experience this on Mac OS X 13.5.1
python version: 3.10
vips version: 8.12.2
pyvips version: 2.2.1
numpy version: 1.23.1

My solution is similar to emzo, but I solve it this way:
If the ls array has any values that are below the abs(step), then I set it to step. This gets rid of any zero values as well.

So after line 64

I do:

        ### DEBUG: Why is there a black line on some images? ###
        # it appears to be whenever the value is below abs(step)
        # boolArray = (ls >-step)&(ls < step)
        # print(ls[boolArray])

        ls[(ls>-step)&(ls<= 0)] = -step
        ls[(ls<step)&(ls>= 0)] = step

Unfortunately, while this does remove the 1px black line on py, it creates very small stitching artifact when other tiles are put together because pixels are technically being removed. Emzo's solution also has this problem.

I find it very perplexing and worked on this problem a lot. But I am not an expert in these libraries. I find it pretty alarming that it is not consistently reproducable on different operating systems (though indeed, in my own environment the problem happens every time). To me, this is a pyvips bug that it doesnt happen consistently on windows vs os x?

One thought I had is it is related to initial face resize (similar issue) or maybe a displacement caused by rotate (do we rotate to generate images?) (similar issue) ?

Indeed by the way, passing centre arguments, etc has no effect. On main branch (my fix not applied), I did observe that the line is slightly reduced when I pass in an odd sized -f 2049 versus -f 2048 but still does exist.

[emzo]

I tried on PC, and didn't experience any problems, so this is definitely a MacOS problem. Haven't tried on Linux yet.

…

On Fri, 15 Sep 2023, 4:27 pm Clayton Rothschild, ***@***.***> wrote: I also experience this on Mac OS X 13.5.1 python version: 3.10 vips version: 8.12.2 pyvips version: 2.2.1 numpy version: 1.23.1 My solution is similar to emzo, but I solve it this way: If the ls array has any values that are below the abs(step), then I set it to step. This gets rid of any zero values as well. So after line 64 <https://github.com/indus/kubi/blob/75a3265f8da07e42dd56fba2b10af0aa2273aa1c/src/kubi/kubi.py#L64> I do: ### DEBUG: Why is there a black line on some images? ### # it appears to be whenever the value is below abs(step) # boolArray = (ls >-step)&(ls < step) # print(ls[boolArray]) ls[(ls>-step)&(ls<= 0)] = -step ls[(ls<step)&(ls>= 0)] = step Unfortunately, while this does remove the 1px black line on py, it creates very small stitching artifact when other tiles are put together because pixels are technically being removed. Emzo's solution also has this problem. I find it very perplexing and worked on this problem a lot. But I am not an expert in these libraries. I find it pretty alarming that it is not consistently reproducable on different operating systems (though indeed, in my own environment the problem happens every time). To me, this is a pyvips bug that it doesnt happen consistently on windows vs os x? One thought I had is it is related to initial face resize (similar issue <libvips/libvips#1476>) or maybe a displacement caused by rotate (do we rotate to generate images?) (similar issue <libvips/libvips#1051>) ? Indeed by the way, passing centre arguments, etc has no effect. On main branch (my fix not applied), I did observe that the line is slightly reduced when I pass in an odd sized -f 2049 versus -f 2048 but still does exist. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKJ4ZKNAKVW4CCM6HMPTX2RQVZANCNFSM54ITI5DQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[claytonrothschild]

I would be glad to open issue on libvips if I knew how to phrase it.

[claytonrothschild]

Could anyone help me describe the issue succinctly so I can open the issue on libvips?

[indus]

@claytonrothschild I think an example may be better?!
Besides that, have you tried to modify the code in a way that the mapim option extend is set to copy? Because - as you said - this sounds similar: libvips/libvips#1476

[emzo]

@indus the mapim option extend essentially just instruct how to "fill in the gaps". Looking here COPY "takes the value of the nearest edge pixel", whereas BLACK (the default I assume) "makes the new pixels black".

I guess you're right that changing the extend option would effectively remove the black line, however, I think it's not the right way to do it, as it's just copying (or doubling up) the last column (or row?) of pixels in the array. In the issue you referenced libvips/libvips#1476, the author of libvips himself suggests "You might also want to consider using --idx 0.5 --idy 0.5 to enable centre sampling. It usually gives more natural results for upsampling."

"Centre sampling" is effectively what PR #4 is attempting to achieve, so that there is no need to fill any missing pixels in. However, I still have no idea why this in only happening on MacOS and not on Windows 🤷‍♂️

[emzo]

Unfortunately, while this does remove the 1px black line on py, it creates very small stitching artifact when other tiles are put together because pixels are technically being removed. Emzo's solution also has this problem.

@claytonrothschild I'm glad I'm not the only one facing this issue 😅

You mention my solution also has this problem? Do you have an example showing this, since I was under the impression that it didn't, although I may be wrong of course 😆

[piggz]

Ive also got this issue. I have source images that are 2:1 7064x3532 equirectangular photosphere images. Im trying to output them to google format cubemap for use in marzipano, and it works well, except for a 1px line on the rear face (b) along the centre of the image. Seems to be present on all the zoom levels, just in this location. I tried the 2 fixes above but neither seemed to work.

im on windows 10, latest vips, python 3.12.4

[piggz]

I resampled the source image to 8192x4096 and still get a black line on eg b/3/3/3.jpg

command is
kubi -s 4096 -co tile_size=512 -co depth=onetile -co overlap=0 layout=google -co suffix=jpg[Q=80] -f r l u d f b *.jpg tiles\tiles.dz

PS it would be nice to get some better naming of the output foldername, using just the basename of the source image as the folder name, followed by face, zoom, y, x folders

[indus]

@piggz the comments in this issue suggest that it is MacOS problem. I don't own a Mac so I don't have a chance to validate any fixes. But if somebody with a mac comes up with changes to the code that fixes or mitigates the issue without effects to the output on other systems I'm happy to merge.

regarding the folders and names the options are a bit limited. You can try to change the create-option layout :

-co layout=google (options are dz, zoomify, google, iiif)

They are defined in vips and just used by kubi. I've jsut seen that there now 2 more: https://www.libvips.org/API/current/VipsForeignSave.html#VipsForeignDzLayout. They should be valid options without any code changes. Maybe one of them brings you closer to what you need.

[indus]

@piggz oh I've just noticed that you had this issue on a PC. 🤔
That makes me really wonder what makes the difference now. Can you provide more details on you system? ... and maybe the input file?

[piggz]

i will try an recreate it locally with a generic input file, I dont want to use anything from work.

The system is pretty unexciting, i installed anaconda3 a week or so ago, created a new env for kubi, did a conda install pip and git, and then a pip install for kubi with the url, and manually installed the latest vips for windows and added it to the PATH.

The line is very visible in the final jpeg in the 1px wide column that would be the centre of the back face of the cube.

[piggz]

@indus Im also able to recreate this on my home Linux laptop.

Source file : https://live.staticflickr.com/65535/53916609173_2366106599_4k.jpg
Command: kubi -s 4096 -co tile_size=512 -co depth=onetile -co overlap=0 -co layout=google -co suffix=.jpg[Q=80] -f r l u d f b 53916609173_2366106599_4k.jpg cm.dz

Attached one of the output images with a black column

My system is openSuse tumbleweed
Python 3.11.9
libvips 8.15.1

The black line is clearly visible here

[indus]

I've just released a new version: https://github.com/indus/kubi/releases/tag/v0.1.6
That should fix the issue with the black line. For this change to work kubi now requires libvips v8.13.0 or above.

I've checked the changes for @piggz input and no longer find a black line:

(... a JPEG with compression may not be the best test case 😁 )

I am optimistic (but not sure) about all the other cases... feedback is welcome.

[piggz]

Fantastic, will try it on a bunch of images tomorrow.

[piggz]

Im happy with these results, generated >8000 images now :)