nss_winbind: fix hang on Solaris on big groups

The problem with large groups on Solaris in the the NSS winbind module is
Solaris wants the return value to be NSS_UNAVAIL if the buffer given is too
small for getgrnam_r.  The current code return NSS_TRYAGAIN which causes
Solaris/Illumos to loop without trying to resize the buffer.

Thanks to  Nathan Huff <[email protected]> for finding this out.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10365

Signed-off-by: Bjoern Jacke <[email protected]>
Reviewed-by: Ralph Böhme <[email protected]>

nsswitch/winbind_nss_solaris.c

@@ -259,6 +259,9 @@ _nss_winbind_getgrnam_solwrap(nss_backend_t* be, void* args)
 	if(ret == NSS_STATUS_SUCCESS)
 		NSS_ARGS(args)->returnval = (void*) result;
 
+	if (NSS_ARGS(args)->erange == ERANGE && ret == NSS_STATUS_TRYAGAIN)
+		return NSS_STATUS_UNAVAIL;
+
 	return ret;
 }
 
@@ -278,6 +281,9 @@ _nss_winbind_getgrgid_solwrap(nss_backend_t* be, void* args)
 	if(ret == NSS_STATUS_SUCCESS)
 		NSS_ARGS(args)->returnval = (void*) result;
 
+	if (NSS_ARGS(args)->erange == ERANGE && ret == NSS_STATUS_TRYAGAIN)
+		return NSS_STATUS_UNAVAIL;
+
 	return ret;
 }