error calculating remaining bytes when reading from text lob stored in value_image

[derivativerobmc]

I ran into a problem trying to read an entire lob block after it had been loaded into the scanner value image from a previous get lob size call. I tracked down the issue to line 1925 in ion_reader_text.c where it updates the length of the remaining bytes.

The line currently reads:
__text->_scanner.value_image.length -= remaining;

but I think it should probably be setting the length to be equal to 'remaining' or just subtracting the bytes written.

It seems to work as I would expect with these changes in my local copy.

[dlurton]

Can you be a little more specific about what type of problem you encountered? I wrote the tests below which read from an Ion-text BLOB and a CLOB and did not encounter any issues. Would it be possible for you to modify one or both of these to reproduce the issue you're experiencing?

TEST(IonTextClob, CanReadClobAfterReadingSize) {
    const char *ion_text = "{{ \"This is a CLOB of text.\" }}";
    const size_t EXPECTED_LEN = 23;
    hREADER reader;

    ION_ASSERT_OK(ion_test_new_text_reader(ion_text, &reader));

    ION_TYPE type;
    ION_ASSERT_OK(ion_reader_next(reader, &type));
    ASSERT_EQ(tid_CLOB, type);

    SIZE lob_size;
    ION_ASSERT_OK(ion_reader_get_lob_size(reader, &lob_size));
    ASSERT_EQ(EXPECTED_LEN, lob_size);

    BYTE *bytes = (BYTE*)malloc(lob_size + 1);
    memset(bytes, 0, lob_size + 1);
    SIZE bytes_read;
    ION_ASSERT_OK(ion_reader_read_lob_bytes(reader, bytes, lob_size, &bytes_read));
    ASSERT_EQ(EXPECTED_LEN, bytes_read);
    char *lob_text = (char*)bytes;
    ASSERT_EQ(strcmp("This is a CLOB of text.", lob_text), 0);
    free(bytes);

    ION_ASSERT_OK(ion_reader_close(reader));
}


TEST(IonTextBlob, CanReadBlobAfterReadingSize) {
    const char *ion_text = "{{ VGhpcyBpcyBhIEJMT0Igb2YgdGV4dC4= }}";
    const size_t EXPECTED_LEN = 23;
    hREADER reader;

    ION_ASSERT_OK(ion_test_new_text_reader(ion_text, &reader));

    ION_TYPE type;
    ION_ASSERT_OK(ion_reader_next(reader, &type));
    ASSERT_EQ(tid_BLOB, type);

    SIZE lob_size;
    ION_ASSERT_OK(ion_reader_get_lob_size(reader, &lob_size));
    ASSERT_EQ(EXPECTED_LEN, lob_size);

    BYTE *bytes = (BYTE*)malloc(lob_size + 1);
    memset(bytes, 0, lob_size + 1);
    SIZE bytes_read;
    ION_ASSERT_OK(ion_reader_read_lob_bytes(reader, bytes, lob_size, &bytes_read));
    ASSERT_EQ(EXPECTED_LEN, bytes_read);
    char *lob_text = (char*)bytes;
    ASSERT_EQ(strcmp("This is a BLOB of text.", lob_text), 0);
    free(bytes);

    ION_ASSERT_OK(ion_reader_close(reader));
}

[derivativerobmc]

sure, my issue was that it wasn't reading the entire lob because the length was getting reduced each loop (using partial read) by more than what was read i.e. it was returning zero bytes read before it had actually finished

I'll give your tests a check as soon as I can; however, at first glance I'm not sure if either would catch it though, because it only occurs when doing partial reads in a loop.

[derivativerobmc]

yeah, neither of those tests trigger the code where I was having issues with. I made a quick modification that showcases the problem by reading the clob in chunks of 5 bytes using the read_lob_partial_bytes function;

TEST(IonTextClob, CanReadClobAfterReadingSize) {
    const char* ion_text = "{{ \"This is a CLOB of text.\" }}";
    const size_t EXPECTED_LEN = 23;
    hREADER reader;

    ION_ASSERT_OK(ion_test_new_text_reader(ion_text, &reader));

    ION_TYPE type;
    ION_ASSERT_OK(ion_reader_next(reader, &type));
    ASSERT_EQ(tid_CLOB, type);

    SIZE lob_size;
    ION_ASSERT_OK(ion_reader_get_lob_size(reader, &lob_size));
    ASSERT_EQ(EXPECTED_LEN, lob_size);

    BYTE* bytes = (BYTE*)malloc(lob_size + 1);
    memset(bytes, 0, lob_size + 1);
    SIZE bytes_read, total_bytes_read = 0;
    //ION_ASSERT_OK(ion_reader_read_lob_bytes(reader, bytes, lob_size, &bytes_read));
    
    const size_t READ_SIZE = 5;
    do
    {
        ION_ASSERT_OK(ion_reader_read_lob_partial_bytes(reader, &bytes[total_bytes_read], READ_SIZE, &bytes_read));
        total_bytes_read += bytes_read;
    } while (bytes_read > 0);

    ASSERT_EQ(EXPECTED_LEN, total_bytes_read);
    char* lob_text = (char*)bytes;
    ASSERT_EQ(strcmp("This is a CLOB of text.", lob_text), 0);
    free(bytes);

    ION_ASSERT_OK(ion_reader_close(reader));
}

[dlurton]

Now that I can work with. Thanks. I'm digging further into it now.