WIP on website+docs (ClickHouse#3668)

* CLICKHOUSE-4063: less manual html @ index.md

* CLICKHOUSE-4063: recommend markdown="1" in README.md

* CLICKHOUSE-4003: manually purge custom.css for now

* CLICKHOUSE-4064: expand <details> before any print (including to pdf)

* CLICKHOUSE-3927: rearrange interfaces/formats.md a bit

* CLICKHOUSE-3306: add few http headers

* Remove copy-paste introduced in ClickHouse#3392

* Hopefully better chinese fonts ClickHouse#3392

* get rid of tabs @ custom.css

* Apply comments and patch from ClickHouse#3384

* Add jdbc.md to ToC and some translation, though it still looks badly incomplete

* minor punctuation

* Add some backlinks to official website from mirrors that just blindly take markdown sources

* Do not make fonts extra light

* find . -name '*.md' -type f | xargs -I{} perl -pi -e 's//g' {}

* find . -name '*.md' -type f | xargs -I{} perl -pi -e 's/ sql/g' {}

* Remove outdated stuff from roadmap.md

* Not so light font on front page too

* Refactor Chinese formats.md to match recent changes in other languages

* Update some links on front page

* Remove some outdated comment

* Add twitter link to front page

* More front page links tuning

* Add Amsterdam meetup link

* Smaller font to avoid second line

* Add Amsterdam link to README.md

* Proper docs nav translation

* Back to 300 font-weight except Chinese

* fix docs build

* Update Amsterdam link

* remove symlinks

* more zh punctuation

* apply lost comment by @zhang2014

* Apply comments by @zhang2014 from ClickHouse#3417

* Remove Beijing link

* rm incorrect symlink

* restore content of docs/zh/operations/table_engines/index.md

* CLICKHOUSE-3751: stem terms while searching docs

* CLICKHOUSE-3751: use English stemmer in non-English docs too

* CLICKHOUSE-4135 fix

* Remove past meetup link

* Add blog link to top nav

* Add ContentSquare article link

* Add form link to front page + refactor some texts

* couple markup fixes

* minor

* Introduce basic ODBC driver page in docs

* More verbose 3rd party libs disclaimer

* Put third-party stuff into a separate folder

* Separate third-party stuff in ToC too

* Update links

* Move stuff that is not really (only) a client library into a separate page

* Add clickhouse-hdfs-loader link

* Some introduction for "interfaces" section

* Rewrite tcp.md

* http_interface.md -> http.md

* fix link

* Remove unconvenient error for now

* try to guess anchor instead of failing

* remove symlink

* Remove outdated info from introduction

* remove ru roadmap.md

* replace ru roadmap.md with symlink

* Update roadmap.md

* lost file

* Title case in toc_en.yml

* Sync "Functions" ToC section with en

* Remove reference to pretty old ClickHouse release from docs

* couple lost symlinks in fa

docs/en/interfaces/index.md

@@ -2,7 +2,19 @@
 
 # Interfaces
 
-To explore the system's capabilities, download data to tables, or make manual queries, use the clickhouse-client program.
+ClickHouse provides two network interfaces (both can be optionally wrapped in TLS for additional security):
 
+* [HTTP](http.md), which is documented and easy to use directly.
+* [Native TCP](tcp.md), which has less overhead.
+
+In most cases it is recommended to use appropriate tool or library instead of interacting with those directly. Officially supported by Yandex are the following:
+* [Command-line client](cli.md)
+* [JDBC driver](jdbc.md)
+* [ODBC driver](odbc.md)
+
+There are also a wide range of third-party libraries for working with ClickHouse:
+* [Client libraries](third-party/client_libraries.md)
+* [Integrations](third-party/integrations.md)
+* [Visual interfaces](third-party/gui.md)
 
 [Original article](https://clickhouse.yandex/docs/en/interfaces/) <!--hide-->

docs/en/interfaces/jdbc.md

@@ -1,7 +1,7 @@
 # JDBC Driver
 
 - [Official driver](https://github.com/yandex/clickhouse-jdbc).
-- Third-party driver from [ClickHouse-Native-JDBC](https://github.com/housepower/ClickHouse-Native-JDBC).
+- Third-party driver [ClickHouse-Native-JDBC](https://github.com/housepower/ClickHouse-Native-JDBC).
 
 
 [Original article](https://clickhouse.yandex/docs/en/interfaces/jdbc/) <!--hide-->

docs/en/interfaces/odbc.md

@@ -0,0 +1,6 @@
+# ODBC Driver
+
+- [Official driver](https://github.com/yandex/clickhouse-odbc).
+
+
+[Original article](https://clickhouse.yandex/docs/en/interfaces/odbc/) <!--hide-->

docs/en/interfaces/tcp.md

@@ -1,6 +1,5 @@
 # Native Interface (TCP)
 
-The native interface is used in the "clickhouse-client" command-line client for interaction between servers with distributed query processing, and also in C++ programs. We will only cover the command-line client.
-
+The native protocol is used in the [command-line client](cli.md), for interserver communication during distributed query processing, and also in other C++ programs. Unfortunately, native ClickHouse protocol does not have formal specification yet, but it can be reverse engineered from ClickHouse source code (starting [around here](https://github.com/yandex/ClickHouse/tree/master/dbms/src/Client)) and/or by intercepting and analyzing TCP traffic.
 
 [Original article](https://clickhouse.yandex/docs/en/interfaces/tcp/) <!--hide-->

docs/en/interfaces/third-party_client_libraries.md → docs/en/interfaces/third-party/client_libraries.md

@@ -1,10 +1,10 @@
-# Libraries from Third-party Developers
+# Client Libraries from Third-party Developers
 
-We have not tested the libraries listed below.
+!!! warning "Disclaimer"
+    Yandex does **not** maintain the libraries listed below and haven't done any extensive testing to ensure their quality.
 
 - Python
     - [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)
-    - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse)
     - [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver)
     - [clickhouse-client](https://github.com/yurial/clickhouse-client)
     - [aiochclient](https://github.com/maximdanilchenko/aiochclient)
@@ -30,23 +30,21 @@ We have not tested the libraries listed below.
 - R
     - [clickhouse-r](https://github.com/hannesmuehleisen/clickhouse-r)
     - [RClickhouse](https://github.com/IMSMWU/RClickhouse)
+- Java
+    - [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java)
 - Scala
     - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client)
-- .NET
+- Kotlin
+    - [AORM](https://github.com/TanVD/AORM)
+- C#
     - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net)
     - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net)
-    - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations)
 - C++
     - [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp/)
 - Elixir
     - [clickhousex](https://github.com/appodeal/clickhousex/)
-    - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto)
-- Java
-    - [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java)
-- Kotlin
-    - [AORM](https://github.com/TanVD/AORM)
 - Nim
     - [nim-clickhouse](https://github.com/leonardoce/nim-clickhouse)
 
 
-[Original article](https://clickhouse.yandex/docs/en/interfaces/third-party_client_libraries/) <!--hide-->
+[Original article](https://clickhouse.yandex/docs/en/interfaces/third-party/client_libraries/) <!--hide-->

docs/en/interfaces/third-party/integrations.md

@@ -0,0 +1,25 @@
+# Integration Libraries from Third-party Developers
+
+!!! warning "Disclaimer"
+    Yandex does **not** maintain the libraries listed below and haven't done any extensive testing to ensure their quality.
+
+- Python
+    - [SQLAlchemy](https://www.sqlalchemy.org)
+        - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse) (uses [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm))
+- Java
+    - [Hadoop](http://hadoop.apache.org)
+        - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader) (uses [JDBC](../jdbc.md))
+- Scala
+    - [Akka](https://akka.io)
+        - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client)
+- C#
+    - [ADO.NET](https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-overview)
+        - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net)
+        - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net)
+        - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations)
+- Elixir
+    - [Ecto](https://github.com/elixir-ecto/ecto)
+        - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto)
+
+
+[Original article](https://clickhouse.yandex/docs/en/interfaces/third-party/integrations/) <!--hide-->

docs/en/operations/settings/permissions_for_queries.md

@@ -33,7 +33,7 @@ See [above](#permissions_for_queries) for the division of queries into groups.
 
 After setting `readonly = 1`, a user can't change `readonly` and `allow_ddl` settings in the current session.
 
-When using the `GET` method in the [HTTP interface](../../interfaces/http_interface.md#http_interface), `readonly = 1` is set automatically. To modify data use the `POST` method.
+When using the `GET` method in the [HTTP interface](../../interfaces/http.md#http_interface), `readonly = 1` is set automatically. To modify data use the `POST` method.
 
 <a name="settings_allow_ddl"></a>
 

docs/en/operations/table_engines/custom_partitioning_key.md

@@ -2,8 +2,6 @@
 
 # Custom Partitioning Key
 
-Starting with version 1.1.54310, you can create tables in the MergeTree family with any partitioning expression (not only partitioning by month).
-
 The partition key can be an expression from the table columns, or a tuple of such expressions (similar to the primary key). The partition key can be omitted. When creating a table, specify the partition key in the ENGINE description with the new syntax:
 
 ```

docs/en/roadmap.md

@@ -4,11 +4,23 @@
 
 - JOIN syntax compatible with SQL standard:
     - Mutliple `JOIN`s in single `SELECT`
+- Protobuf and Parquet input and output formats
+
+## Q1 2019
+
+- Import/export from HDFS and S3
+- Lower metadata size in ZooKeeper
+- Adaptive index granularity for MergeTree engine family
+
+## Q2 2019
 
 - JOIN execution improvements:
     - Distributed join not limited by memory
-
-- Protobuf and Parquet input and output formats
 - Resource pools for more precise distribution of cluster capacity between users
 
+## Q3 2019
+
+- Fine-grained authorization
+- Integration with external authentication services
+
 [Original article](https://clickhouse.yandex/docs/en/roadmap/) <!--hide-->

docs/fa/interfaces/index.md

@@ -2,9 +2,22 @@
 
 <div dir="rtl" markdown="1">
 
-# Interface ها
+# رابط ها
 
-برای کشف قابلیت های سیستم، دانلو داده ها به جداول، یا ساخت query های دستی، از برنامه clikhouse-client استفاده کنید.
+ClickHouse دو اینترفیس شبکه را فراهم می کند (هر دو می توانند به صورت اختیاری در TLS برای امنیت اضافی پیچیده شوند):
+
+* [HTTP](http.md), که مستند شده و به راحتی به طور مستقیم استفاده می شود.
+* [بومی TCP](tcp.md), که دارای سربار کمتر است.
+
+اگرچه در بیشتر موارد توصیه می شود از ابزار یا کتابخانه مناسب استفاده کنید تا به طور مستقیم با آن ها ارتباط برقرار نکنید. به طور رسمی توسط یانداکس پشتیبانی می شوند عبارتند از:
+* [خط فرمان خط](cli.md)
+* [راننده JDBC](jdbc.md)
+* [راننده ODBC](odbc.md)
+
+همچنین برای کار با ClickHouse طیف گسترده ای از کتابخانه های شخص ثالث وجود دارد:
+* [کتابخانه های مشتری](third-party/client_libraries.md)
+* [ادغام](third-party/integrations.md)
+* [رابط های بصری](third-party/gui.md)
 
 </div>
 [مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/) <!--hide-->

docs/fa/interfaces/jdbc.md

@@ -5,10 +5,8 @@
 درایور رسمی JDBC برای ClickHouse وجود دارد. برای اطلاعات بیشتر [اینجا](https://github.com/yandex/clickhouse-jdbc) را ببینید.
 
 
-JDBC drivers implemented by other organizations:
 درایور JDBC توسط سازمان های دیگر اجرا می شوند.
 
-
 - [ClickHouse-Native-JDBC](https://github.com/housepower/ClickHouse-Native-JDBC)
 
 </div>

docs/fa/interfaces/odbc.md

@@ -0,0 +1,7 @@
+<div dir="rtl" markdown="1">
+# ODBC درایور
+
+درایور رسمی ODBC برای ClickHouse وجود دارد. برای اطلاعات بیشتر [اینجا](https://github.com/yandex/clickhouse-odbc) را ببینید.
+
+</div>
+[مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/odbc/) <!--hide-->

docs/fa/interfaces/tcp.md

@@ -1,8 +1,8 @@
 <div dir="rtl" markdown="1">
 
-# Native interface (TCP)
+# رابط بومی (TCP)
 
-native interface در محیط  ترمینال "clickhouse-client" برای تعامل بین سرور با پردازش query توزیع شده مورد استفاده قرار می گیرد. همچنین native interface در برنامه های C++ مورد استفاده قرار می گیرد. ما فقط کلاینت command-line را پوشش میدیم.
+پروتکل بومی در [خط فرمان خط] (cli.md)، برای برقراری ارتباط بین سرور در طی پردازش پرس و جو توزیع شده، و همچنین در سایر برنامه های C ++ استفاده می شود. متاسفانه، پروتکل ClickHouse بومی هنوز مشخصات رسمی ندارد، اما می توان آن را از کد منبع ClickHouse (شروع [از اینجا](https://github.com/yandex/ClickHouse/tree/master/dbms/src/Client)) و / یا با رهگیری و تجزیه و تحلیل ترافیک TCP.
 
 </div>
 [مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/tcp/) <!--hide-->

docs/fa/interfaces/third-party_client_libraries.md → docs/fa/interfaces/third-party/client_libraries.md

@@ -1,12 +1,12 @@
 <div dir="rtl" markdown="1">
 
-# کتابخانه های توسعه دهندگان third-party
+# کتابخانه های مشتری شخص ثالث
 
-کتابخانه هایی برای کار با ClickHouse وجود دارد:
+!!! warning "سلب مسئولیت"
+    Yandex نه حفظ کتابخانه ها در زیر ذکر شده و نشده انجام هر آزمایش های گسترده ای برای اطمینان از کیفیت آنها.
 
 - Python
     - [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)
-    - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse)
     - [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver)
     - [clickhouse-client](https://github.com/yurial/clickhouse-client)
 - PHP
@@ -31,21 +31,23 @@
 - R
     - [clickhouse-r](https://github.com/hannesmuehleisen/clickhouse-r)
     - [RClickhouse](https://github.com/IMSMWU/RClickhouse)
-- .NET
+- Java
+    - [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java)
+- Scala
+    - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client)
+- Kotlin
+    - [AORM](https://github.com/TanVD/AORM)
+- C#
     - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net)
     - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net)
-    - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations)
 - C++
     - [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp/)
 - Elixir
     - [clickhousex](https://github.com/appodeal/clickhousex/)
-    - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto)
-- Java
-    - [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java)
 - Nim
     - [nim-clickhouse](https://github.com/leonardoce/nim-clickhouse)
 
 ما این کتابخانه ها را تست نکردیم. آنها به صورت تصادفی انتخاب شده اند.
 
 </div>
-[مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/third-party_client_libraries/) <!--hide-->
+[مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/third-party/client_libraries/) <!--hide-->

docs/fa/interfaces/third-party/integrations.md

@@ -0,0 +1,27 @@
+<div dir="rtl" markdown="1">
+
+# کتابخانه ادغام ثالث
+
+!!! warning "سلب مسئولیت"
+    Yandex نه حفظ کتابخانه ها در زیر ذکر شده و نشده انجام هر آزمایش های گسترده ای برای اطمینان از کیفیت آنها.
+
+- Python
+    - [SQLAlchemy](https://www.sqlalchemy.org)
+        - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse) (uses [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm))
+- Java
+    - [Hadoop](http://hadoop.apache.org)
+        - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader) (uses [JDBC](../jdbc.md))
+- Scala
+    - [Akka](https://akka.io)
+        - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client)
+- C#
+    - [ADO.NET](https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-overview)
+        - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net)
+        - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net)
+        - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations)
+- Elixir
+    - [Ecto](https://github.com/elixir-ecto/ecto)
+        - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto)
+
+</div>
+[مقاله اصلی](https://clickhouse.yandex/docs/fa/interfaces/third-party/integrations/) <!--hide-->

docs/fa/introduction/features_considered_disadvantages.md

@@ -3,7 +3,7 @@
 # ویژگی های از ClickHouse که می تواند معایبی باشد.
 
 1. بدون پشتیبانی کامل از تراکنش
-2. عدم توانایی در تغییر و یا حذف داده های insert شده با rate بالا و latency کم. روشی برای پاک کردن دسته ای داده ها و یا مطابق با قوانین [GDPR](https://gdpr-info.eu) وجود دارد. بروزرسانی دسته ای از July 2018 در حال توسعه می باشد. 
+2. عدم توانایی برای تغییر و یا حذف داده های در حال حاضر وارد شده با سرعت بالا و تاخیر کم. برای پاک کردن و یا اصلاح داده ها، به عنوان مثال برای پیروی از [GDPR](https://gdpr-info.eu)، دسته ای پاک و به روزرسانی وجود دارد.حال توسعه می باشد.
 3. Sparse index باعث می شود ClickHouse چندان مناسب اجرای پرسمان های point query برای دریافت یک ردیف از داده ها با استفاده از کلید آنها نباشد.
 
 </div>

docs/fa/query_language/functions/functions_for_nulls.md

@@ -0,0 +1 @@
+../../../en/query_language/functions/functions_for_nulls.md

docs/fa/query_language/functions/geo.md

@@ -0,0 +1 @@
+../../../en/query_language/functions/geo.md

docs/redirects.txt

@@ -116,3 +116,6 @@ table_functions/numbers.md query_language/table_functions/numbers.md
 table_functions/remote.md query_language/table_functions/remote.md
 query_language/queries.md query_language.md
 introduction/possible_silly_questions.md faq/general.md
+interfaces/third-party_client_libraries.md interfaces/third-party/client_libraries.md 
+interfaces/third-party_gui.md interfaces/third-party/gui.md 
+interfaces/http_interface.md interfaces/http.md

docs/ru/interfaces/index.md

@@ -2,6 +2,19 @@
 
 # Интерфейсы
 
-Для изучения возможностей системы, загрузки данных в таблицы, ручных запросов, используйте программу clickhouse-client.
+ClickHouse предоставляет два сетевых интерфейса (оба могут быть дополнительно обернуты в TLS для дополнительной безопасности):
+
+* [HTTP](http.md), который задокументирован и прост для использования напрямую;
+* [Native TCP](tcp.md), который имеет меньше накладных расходов.
+
+В большинстве случаев рекомендуется использовать подходящий инструмент или библиотеку, а не напрямую взаимодействовать с ClickHouse по сути. Официально поддерживаемые Яндексом:
+* [Консольный клиент](cli.md);
+* [JDBC-драйвер](jdbc.md);
+* [ODBC-драйвер](odbc.md).
+
+Существует также широкий спектр сторонних библиотек для работы с ClickHouse:
+* [Клиентские библиотеки](third-party/client_libraries.md);
+* [Библиотеки для интеграции](third-party/integrations.md);
+* [Визуальные интерфейсы](third-party/gui.md).
 
 [Оригинальная статья](https://clickhouse.yandex/docs/ru/interfaces/) <!--hide-->

docs/ru/interfaces/odbc.md

@@ -0,0 +1,5 @@
+# ODBC-драйвер
+
+- [Официальный драйвер](https://github.com/yandex/clickhouse-jdbc).
+
+[Оригинальная статья](https://clickhouse.yandex/docs/ru/interfaces/odbc/) <!--hide-->

docs/ru/interfaces/tcp.md

@@ -1,5 +1,5 @@
 # Родной интерфейс (TCP)
 
-Родной интерфейс используется в клиенте командной строки clickhouse-client, при межсерверном взаимодействии для распределённой обработки запроса, а также в программах на C++. Будет рассмотрен только клиент командной строки.
+Нативный протокол используется в [клиенте командной строки](cli.md), для взаимодействия между серверами во время обработки распределенных запросов, а также в других программах на C++. К сожалению, у родного протокола ClickHouse пока нет формальной спецификации, но в нем можно разобраться с использованием исходного кода ClickHouse (начиная с [примерно этого места](https://github.com/yandex/ClickHouse/tree/master/dbms/src/Client)) и/или путем перехвата и анализа TCP трафика.
 
 [Оригинальная статья](https://clickhouse.yandex/docs/ru/interfaces/tcp/) <!--hide-->