Strip sitemap entries (langchain-ai#2132)

Loading this sitemap didn't work for me https://www.alzallies.com/sitemap.xml Changing this fixed it and it seems like a good idea to do it in general. Integration tests pass
kevinNejad · Mar 29, 2023 · 4ab66c4 · 4ab66c4
1 parent 27f8078
commit 4ab66c4
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/langchain/document_loaders/sitemap.py b/langchain/document_loaders/sitemap.py
@@ -58,7 +58,7 @@ def load(self) -> List[Document]:
 
         els = self.parse_sitemap(soup)
 
-        results = self.scrape_all([el["loc"] for el in els if "loc" in el])
+        results = self.scrape_all([el["loc"].strip() for el in els if "loc" in el])
 
         return [
             Document(