Add some labels from target domain hurts performance

[YiruS]

Hi there,

First of all, nice work!

I'd like to add some labels from target domain for training (pick 100 shapes from target domain to assign the GT labels for training). However, it turns out the classification accuracy drops from ~64% to ~59%. Below is my code snippet:

        pred_s1, pred_s2 = model(data)
        pred_t1, pred_t2 = model(data_t, constant=cons, adaptation=True)

        # Classification loss

        loss_s1 = criterion(pred_s1, label)
        loss_s2 = criterion(pred_s2, label)
        loss_t1 = criterion(pred_t1, label_t)
        loss_t2 = criterion(pred_t2, label_t)

        # Adversarial loss

        loss_adv = - 1 * discrepancy(pred_t1, pred_t2)

        loss_s = loss_s1 + loss_s2 + loss_t1 + loss_t2
        loss = args.weight * loss_s + loss_adv

What I don't quite understand is that -- why the performance will drop after adding some real labels from target domain for training? Do you have any insights into this?

Much appreciate your comments!

Best,
Yiru

[canqin001]

Hi Yiru. I have not tested this yet. Did you try all the domain pairs or simply try one pair?

[YiruS]

Hi, I only tried modelnet to ShapeNet. It seems off that from UDA to SDA (supervised domain adaptation), the performance drops 🤔Could you kindly take a look at my code snippet so see if there’s any obvious error? Much appreciated! -Yiru

…

On Fri, Oct 16, 2020 at 6:49 AM canqin001 ***@***.***> wrote: Hi Yiru. I have not tested this yet. Did you try all the domain pairs or simply try one pair? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADMGPCDVBP357CCMW74QPKTSLBFO5ANCNFSM4SSYRRDQ> .

[canqin001]

Hi Yiru. I don't see any bugs in your code. For the SDA setting, I guess the model might be overfitting towards the target domain if few labeled samples are provided. It is a very interesting problem.