[WIP] Development rexview (gempy-project#296)

* refactored write_data_block to write_data_block_header The name of the function was misleading, it does not write the full data block but only the header. A data block consist of a header and the data, depending on the data itself it can have also sub headers. * add function to write material data block * added credential file for rexos Api key * Revert "added credential file for rexos Api key" This reverts commit e8cc48c * added credential file for rexos Api key... again * added api file to .gitignore this prevents that api keys are accidentally added to the repository * added rex_api file * updated rex_api file to use token and secret * added api call to authorize session and retrieve access_|token * added api calls to get user information and to create new projects * safety commit * added api call to create project file * added api calls for ressource management, fixed file upload * wrapper function for api calls * [ENH] AR: Added normals to both sides of the surfaces * added method to create and display rextag * [ENH] WIP Passing gempy colors to rex * [ENH] Both sided colors kind of running * [ENH] Double side layers running * [ENH] Added Test to update to rexos * fixed bug in rex ressource referencing * [TEST] Skip gempy_to_rex tests in Travis * refactoring class to be more explizit seperated rextag top its own class added function to automate upload and create rextag * [DOC] Preparing README.md for GemPy 2.1 * [GIT] - * [TEST] skipping test in travis * [ENH] rex_api.py will write the RexCloud_Api_key.txt if needed * [BUG] Added pyqrcode travis * [ENH] Add error if pyqr code is not installed * [ENH] On rex file export: name layers based on their layername gempy-project#316 [CLN] - Deleted prototype code * [ENH] Minor improvements in rex_api.py/gempy.plot.plot_ar - Surfaces take the name of the gempy model - Project take the name of the gempy model [TEXT] - plot_ar * [ENH] Added the option to rescale vertex coordinates between 0-1 - Change requirements.txt for pandas and qgrid * [BUG] Pandas version * [DOC] Docstring for plot_ar * [BUG] Pandas 1.0.1 breaks gempy! * [BUG] Fix bug when no passing marching_cubes options * Revert "[BUG] Fix bug when no passing marching_cubes options" This reverts commit 5524365 * Revert "[BUG] Fix bug when no passing marching_cubes options" This reverts commit 5524365 * [BUG] Removed ipyvisualization Co-authored-by: Simon Virgo <[email protected]> Co-authored-by: varga <[email protected]>
Lindingdou · Mar 6, 2020 · 78b6111 · 78b6111
1 parent 5080480
commit 78b6111
Show file tree

Hide file tree

Showing 15 changed files with 885 additions and 385 deletions.
diff --git a/.gitignore b/.gitignore
@@ -8,46 +8,7 @@ GeMpy/__pycache__/__init__.cpython-36.pyc
 GeMpy/__pycache__/geomodeller_xml_obj.cpython-36.pyc
 GeMpy/geomodeller_xml_obj.py.bak
 GeMpy/qgrid/__pycache__/
-notebooks/prototype notebooks/Example_4_faults_network.ipynb
-notebooks/prototype notebooks/Example_Peter_Model.ipynb
-notebooks/prototype notebooks/Fabian.vtp
-notebooks/prototype notebooks/Fabian1.vtp
-notebooks/prototype notebooks/Fabian2.vtp
-notebooks/prototype notebooks/Fabian3.vtp
-notebooks/prototype notebooks/Fabian4.vtp
-notebooks/prototype notebooks/FabianModelBigPoints.vtk
-notebooks/prototype notebooks/FabianModelSmallPoints.vtk
-notebooks/prototype notebooks/FabianModel_point.vti
-notebooks/prototype notebooks/FabianModel_point_good_res.vti
-notebooks/prototype notebooks/Fabian_f.vtp
-notebooks/prototype notebooks/Fault.vtk
-notebooks/prototype notebooks/Intrusions.ipynb
-notebooks/prototype notebooks/Layer1.vtk
-notebooks/prototype notebooks/Layer2.vtk
-notebooks/prototype notebooks/Layer3.vtk
-notebooks/prototype notebooks/Layer4.vtk
-notebooks/prototype notebooks/More cave/
-notebooks/prototype notebooks/PF_over_TKC/
-notebooks/prototype notebooks/PerthBasin.vti
-notebooks/prototype notebooks/PerthBasin.vtk
-notebooks/prototype notebooks/PerthBasinOtherColors.vtk
-notebooks/prototype notebooks/Physics.ipynb
-notebooks/prototype notebooks/SandstoneSol.npy
-notebooks/prototype notebooks/TriangleColoredPoints.vtp
-notebooks/prototype notebooks/TriangleSolidColor.vtp
-notebooks/prototype notebooks/Visualization3_vtkBasics.ipynb
-notebooks/prototype notebooks/files fab/
-notebooks/prototype notebooks/geomodels.sqlite
-notebooks/prototype notebooks/legacy/.ipynb_checkpoints/
-notebooks/prototype notebooks/models.npy
-notebooks/prototype notebooks/olaqases.vox
-notebooks/prototype notebooks/preth.npy
-notebooks/prototype notebooks/sandstone.vtk
-notebooks/prototype notebooks/sandstone.vtr
-notebooks/prototype notebooks/simplices.npy
-notebooks/prototype notebooks/vertices.npy
-notebooks/prototype notebooks/voxet_sub.vox
-notebooks/prototype notebooks/withfault.npy
+gempy/addons/RexCloud_Api_key.txt
 docs/source/Notebooks/Erosion.rst
 docs/source/Notebooks/Example_1_Sandstone_files/Example_1_Sandstone_39_0.png
 docs/source/Notebooks/Example_1_Sandstone_files/Example_1_Sandstone_41_0.png

diff --git a/.travis.yml b/.travis.yml
@@ -5,6 +5,8 @@ dist: xenial
 python:
   - '3.7'
 
+env: TRAVIS=true
+
 install:
   - pip install --upgrade setuptools pip
   - pip install cython
@@ -15,7 +17,6 @@ install:
   - pip install vtk
   - pip install pyevtk
   - pip install dataclasses emg3d pyvista panel discretize
-  - pip install nptyping
 
 script:
 

diff --git a/README.md b/README.md
@@ -34,40 +34,6 @@ Check out the documentation either in [gempy.org](https://www.gempy.org/) (bette
 * [References](#ref)
 
 <a name="feat"></a>
-## News
-### GemPy v2.0 release
-
-It has been a long journey since the release of GemPy v1.0. What started as a small library to carry out research
-on uncertainty analysis for structural geology has grown to be used in multiple projects around the world. Carried
-by the community enthusiasm, we commenced a way-longer-than-planned rewritten of the code in order to
-not only be able to fulfill the needs of many of you but also to set the foundations of a package driven by the
-community. For this end, all the logic has been splat into multiple modules, classes and containers limiting
-duplicities and exposing a large mutation api at different levels of abstraction. Hope the work has been worth it.
-
-So long,
-
-Miguel
-
-#### What is new
-- Full redesign of the back-end: much more modular, explicit and avoiding object duplicities to insane levels
-- Topography
-- Onlap, Erosion relations
-- Choose your favourite type of fault: infinite faults, finite faults, faults offsetting faults, faults ending on series
-- Masked marching cubes: this fix the ugly surfaces following the voxels faces
-- All series are fully stored after interpolation
-- Save your model
-- Compile once, modify as much as you want
-- Full integration with qgrid
-- Real time computations via vtk or python-qgrid
-- Adaptive regular grids for geophysics
-- Refactored some legacy names:
-    + formations renamed to surfaces
-    + interfaces renamed to surfaces_points
-- Minor changes:
-    + New colormap and easy way to change the surfaces colors (even integration with widgets!)
-    + The order of the formations will be given by the interpolation itself if the input was wrong
-    + The split between reference and rest surface_points happens in theano. This makes the modification
-    of reference points much easier
 
 
 ## Features

diff --git a/gempy/addons/gempy_to_rexfile.py b/gempy/addons/gempy_to_rexfile.py
@@ -93,7 +93,7 @@ def write_header_block(n_data_blocks, size_data_blocks, version=1, start_data=86
     return block_bytes
 
 
-def write_data_block(size_data, data_id = 1, data_type=3, version_data=1):
+def write_data_block_header(size_data, data_id = 1, data_type=3, version_data=1):
     """
     Function to write a data block header.
 
@@ -118,7 +118,7 @@ def write_data_block(size_data, data_id = 1, data_type=3, version_data=1):
 
 def write_mesh_header(n_vtx_coord,  n_triangles,
                       start_vtx_coord, start_nor_coord, start_tex_coord, start_vtx_colors, start_triangles,
-                      name, material_id=9223372036854775807,
+                      name, material_id=1,  # material_id=9223372036854775807
                       n_nor_coord=0, n_tex_coord=0, n_vtx_colors=0,
                       lod=1, max_lod=1):
     """
@@ -143,6 +143,7 @@ def write_mesh_header(n_vtx_coord,  n_triangles,
 
     """
 
+
     str_size = len(name)
     rest_name_ = ' ' * (74 - str_size)
     full_name = name + rest_name_
@@ -207,51 +208,121 @@ def write_file(bytes, path: str):
     return True
 
 
-def geo_model_to_res(geo_model, path='./gempy_rex'):
+def write_material_data(ka_red=255.0/255, ka_green=255.0/255, ka_blue=255.0/255, ka_texture_ID=9223372036854775807,  # ambient
+                        ks_red=255.0/255, ks_green=255.0/255, ks_blue=255.0/255, ks_texture_ID=9223372036854775807,  # specular
+                        kd_red=255.0/255, kd_green=255.0/255, kd_blue=255.0/255, kd_texture_ID=9223372036854775807,  # diffuse
+                        ns=0.1,  #specular exponent
+                        alpha=1  #opacity
+                        ):
+    """
+    writes a standard material definition block
+
+    Returns: bytes (size:68) representation of the material
+
+    """
+
+    input_ = [(ka_red, 'float32'), (ka_green, 'float32'), (ka_blue, 'float32'), (ka_texture_ID, 'uint64'),
+              (ks_red, 'float32'), (ks_green, 'float32'), (ks_blue, 'float32'), (ks_texture_ID, 'uint64'),
+              (kd_red, 'float32'), (kd_green, 'float32'), (kd_blue, 'float32'), (kd_texture_ID, 'uint64'),
+              (ns,'float32'), (alpha, 'float32')]
+
+    block_bytes = encode(input_)
+    return block_bytes
 
+
+# TODO Move to utils
+def hex_to_rgb(hex):
+    hex = hex.lstrip('#')
+    hlen = len(hex)
+    return tuple(int(hex[i:i + hlen // 3], 16) for i in range(0, hlen, hlen // 3))
+
+
+def geo_model_to_rex(geo_model, path='./gempy_rex'):
+    file_names = []
     mesh_header_size = 128
     file_header_size = 86
     e = 0
-    for ver, tri in zip(geo_model.solutions.vertices, geo_model.solutions.edges):
 
-        colors = np.ones_like(ver) * 200
+    for idx, surface_vals in geo_model.surfaces.df.iterrows():
+        ver = surface_vals['vertices']
+        tri = surface_vals['edges']
+        if tri is np.nan:
+            break
+
+        col = surface_vals['color']
+
+        colors = (np.zeros_like(ver) + hex_to_rgb(col))/255
 
         ver_ = np.copy(ver)
         ver_[:, 2] = ver[:, 1]
         ver_[:, 1] = ver[:, 2]
 
         tri_ = np.copy(tri)
-        tri_[:, 2] = tri[:, 1]
-        tri_[:, 1] = tri[:, 2]
 
         ver_ravel, tri_ravel, n_vtx_coord, n_triangles = mesh_preprocess(ver_, tri_)
-        data_block_size_no_header = 2 * (n_vtx_coord + n_triangles) * 4 + mesh_header_size
-
+        mesh_block_size_no_data_block_header = (2 * n_vtx_coord + n_triangles) * 4 + mesh_header_size
+        material_block_size_no_data_block_header = 68
         # Write header
-        header_bytes = write_header_block(n_data_blocks=1,
-                                          size_data_blocks=data_block_size_no_header + rexDataBlockHeaderSize,
+        n_data_blocks = 3
+        header_bytes = write_header_block(n_data_blocks=n_data_blocks,
+                                          size_data_blocks=mesh_block_size_no_data_block_header +
+                                                           rexDataBlockHeaderSize +
+                                                           material_block_size_no_data_block_header,
                                           start_data=file_header_size)
 
         # Write data block
-        data_bytes = write_data_block(size_data=data_block_size_no_header,
-                                      data_id=1, data_type=3, version_data=1)
+        data_bytes = write_data_block_header(size_data=mesh_block_size_no_data_block_header,
+                                             data_id=1, data_type=3, version_data=1)
 
         # Write mesh block
         mesh_header_bytes = write_mesh_header(n_vtx_coord / 3, n_triangles / 3, n_vtx_colors=n_vtx_coord/3,
                                               start_vtx_coord=mesh_header_size,
                                               start_nor_coord=mesh_header_size + n_vtx_coord * 4,
                                               start_tex_coord=mesh_header_size + n_vtx_coord * 4,
                                               start_vtx_colors=mesh_header_size + n_vtx_coord * 4,
-                                              start_triangles=mesh_header_size + 2 * (n_vtx_coord * 4),
-                                              name='test_a')
+                                              start_triangles=mesh_header_size + 2 *
+                                                              (n_vtx_coord * 4),
+                                              name='test_a', material_id=0)
 
         mesh_block_bytes = write_mesh_coordinates(ver_ravel, tri_ravel, colors=colors.ravel())
 
-        all_bytes = header_bytes + data_bytes + mesh_header_bytes + mesh_block_bytes
+        # Write data block
+        data_bytes_r = write_data_block_header(size_data=mesh_block_size_no_data_block_header,
+                                               data_id=2, data_type=3, version_data=1)
+
+        # One side of the normals
+        tri_[:, 2] = tri[:, 1]
+        tri_[:, 1] = tri[:, 2]
+
+        ver_ravel, tri_ravel, n_vtx_coord, n_triangles = mesh_preprocess(ver_, tri_)
+
+        # Write mesh block
+        mesh_header_bytes_r = write_mesh_header(n_vtx_coord / 3, n_triangles / 3, n_vtx_colors=n_vtx_coord / 3,
+                                              start_vtx_coord=mesh_header_size,
+                                              start_nor_coord=mesh_header_size + n_vtx_coord * 4,
+                                              start_tex_coord=mesh_header_size + n_vtx_coord * 4,
+                                              start_vtx_colors=mesh_header_size + n_vtx_coord * 4,
+                                              start_triangles=mesh_header_size + 2 *
+                                                              (n_vtx_coord * 4),
+                                              name='test_a', material_id=0)
 
-        write_file(all_bytes, path+str(e))
+        mesh_block_bytes_r = write_mesh_coordinates(ver_ravel, tri_ravel, colors=colors.ravel())
+
+        # Write material block
+        material_header_bytes = write_data_block_header(data_type=5, version_data=1, size_data=68, data_id=0)
+        material_bytes = write_material_data()
+
+        all_bytes = header_bytes + data_bytes + mesh_header_bytes + mesh_block_bytes +\
+                                 data_bytes_r + mesh_header_bytes_r + mesh_block_bytes_r +\
+                                 material_header_bytes + material_bytes
+
+        file_name = path+surface_vals['surface']
+        write_file(all_bytes, file_name)
+        file_names.append(file_name+'.rex')
         e += 1
 
+    return file_names
+