omp-low.c (lower_omp_ordered): Add argument to GOMP_SMD_ORDERED_* int…

…ernal calls - 0 if...

gcc/
2015-11-14  Jakub Jelinek  <[email protected]>

	* omp-low.c (lower_omp_ordered): Add argument to GOMP_SMD_ORDERED_*
	internal calls - 0 if ordered simd and 1 for ordered threads simd.
	* tree-vectorizer.c (adjust_simduid_builtins): If GOMP_SIMD_ORDERED_*
	argument is 1, replace it with GOMP_ordered_* call instead of removing
	it.
gcc/c/
2015-11-14  Jakub Jelinek  <[email protected]>

	* c-typeck.c (c_finish_omp_clauses): Don't mark
	GOMP_MAP_FIRSTPRIVATE_POINTER decls addressable.
gcc/cp/
2015-11-14  Jakub Jelinek  <[email protected]>

	* semantics.c (finish_omp_clauses): Don't mark
	GOMP_MAP_FIRSTPRIVATE_POINTER decls addressable.
libgomp/
2015-11-14  Jakub Jelinek  <[email protected]>
	    Aldy Hernandez  <[email protected]>
	    Ilya Verbin  <[email protected]>

	* ordered.c (gomp_doacross_init, GOMP_doacross_post,
	GOMP_doacross_wait, gomp_doacross_ull_init, GOMP_doacross_ull_post,
	GOMP_doacross_ull_wait): For GFS_GUIDED don't divide number of
	iterators or IV by chunk size.
	* parallel.c (gomp_resolve_num_threads): Don't assume that
	if thr->ts.team is non-NULL, then pool must be non-NULL.
	* libgomp-plugin.h (GOMP_PLUGIN_target_task_completion): Declare.
	* libgomp.map (GOMP_PLUGIN_1.1): New symbol version, export
	GOMP_PLUGIN_target_task_completion.
	* Makefile.am (libgomp_la_SOURCES): Add priority_queue.c.
	* Makefile.in: Regenerate.
	* libgomp.h: Shuffle prototypes and forward definitions around so
	priority queues can be defined.
	(enum gomp_task_kind): Add GOMP_TASK_ASYNC_RUNNING.
	(enum gomp_target_task_state): New enum.
	(struct gomp_target_task): Add state, tgt, task and team fields.
	(gomp_create_target_task): Change return type to bool, add
	state argument.
	(gomp_target_task_fn): Change return type to bool.
	(struct gomp_device_descr): Add async_run_func.
	(struct gomp_task): Remove children, next_child, prev_child,
	next_queue, prev_queue, next_taskgroup, prev_taskgroup.
	Add pnode field.
	(struct gomp_taskgroup): Remove children.
	Add taskgroup_queue.
	(struct gomp_team): Change task_queue type to a priority queue.
	(splay_compare): Define inline.
	(priority_queue_offset): New.
	(priority_node_to_task): New.
	(task_to_priority_node): New.
	* oacc-mem.c: Do not include splay-tree.h.
	* priority_queue.c: New file.
	* priority_queue.h: New file.
	* splay-tree.c: Do not include splay-tree.h.
	(splay_tree_foreach_internal): New.
	(splay_tree_foreach): New.
	* splay-tree.h: Become re-entrant if splay_tree_prefix is defined.
	(splay_tree_callback): Define typedef.
	* target.c (splay_compare): Move to libgomp.h.
	(GOMP_target): Don't adjust *thr in any way around running offloaded
	task.
	(GOMP_target_ext): Likewise.  Handle target nowait.
	(GOMP_target_update_ext, GOMP_target_enter_exit_data): Check
	return value from gomp_create_target_task, if false, fallthrough
	as if no dependencies exist.
	(gomp_target_task_fn): Change return type to bool, return true
	if the task should have another part scheduled later.  Handle
	target nowait.
	(gomp_load_plugin_for_device): Initialize async_run.
	* task.c (gomp_init_task): Initialize children_queue.
	(gomp_clear_parent_in_list): New.
	(gomp_clear_parent_in_tree): New.
	(gomp_clear_parent): Handle priorities.
	(GOMP_task): Likewise.
	(priority_queue_move_task_first,
	gomp_target_task_completion, GOMP_PLUGIN_target_task_completion):
	New functions.
	(gomp_create_target_task): Use priority queues.  Change return type
	to bool, add state argument, return false if for async
	{{enter,exit} data,update} constructs no dependencies need to be
	waited for, handle target nowait.  Set task->fn to NULL instead of
	gomp_target_task_fn.
	(verify_children_queue): Remove.
	(priority_list_upgrade_task): New.
	(priority_queue_upgrade_task): New.
	(verify_task_queue): Remove.
	(priority_list_downgrade_task): New.
	(priority_queue_downgrade_task): New.
	(gomp_task_run_pre): Use priority queues.
	Abstract code out to priority_queue_downgrade_task.
	(gomp_task_run_post_handle_dependers): Use priority queues.
	(gomp_task_run_post_remove_parent): Likewise.
	(gomp_task_run_post_remove_taskgroup): Likewise.
	(gomp_barrier_handle_tasks): Likewise.  Handle target nowait target
	tasks specially.
	(GOMP_taskwait): Likewise.
	(gomp_task_maybe_wait_for_dependencies): Likewise.  Abstract code to
	priority-queue_upgrade_task.
	(GOMP_taskgroup_start): Use priority queues.
	(GOMP_taskgroup_end): Likewise.  Handle target nowait target tasks
	specially.  If taskgroup is NULL, and thr->ts.level is 0, act as a
	barrier.
	* taskloop.c (GOMP_taskloop): Handle priorities.
	* team.c (gomp_new_team): Call priority_queue_init.
	(free_team): Call priority_queue_free.
	(gomp_free_thread): Call gomp_team_end if thr->ts.team is artificial
	team created for target nowait in implicit parallel region.
	(gomp_team_start): For nested check, test thr->ts.level instead of
	thr->ts.team != NULL.
	* testsuite/libgomp.c/doacross-3.c: New test.
	* testsuite/libgomp.c/ordered-5.c: New test.
	* testsuite/libgomp.c/priority.c: New test.
	* testsuite/libgomp.c/target-31.c: New test.
	* testsuite/libgomp.c/target-32.c: New test.
	* testsuite/libgomp.c/target-33.c: New test.
	* testsuite/libgomp.c/target-34.c: New test.
liboffloadmic/
2015-11-14  Ilya Verbin  <[email protected]>

	* runtime/offload_host.cpp (task_completion_callback): New
	variable.
	(offload_proxy_task_completed_ooo): Call task_completion_callback.
	(__offload_register_task_callback): New function.
	* runtime/offload_host.h (__offload_register_task_callback): New
	declaration.
	* plugin/libgomp-plugin-intelmic.cpp (offload): Add async_data
	argument, handle async offloading.
	(register_main_image): Call register_main_image.
	(GOMP_OFFLOAD_init_device, get_target_table, GOMP_OFFLOAD_alloc,
	GOMP_OFFLOAD_free, GOMP_OFFLOAD_host2dev, GOMP_OFFLOAD_dev2host,
	GOMP_OFFLOAD_dev2dev) Adjust offload callers.
	(GOMP_OFFLOAD_async_run): New function.
	(GOMP_OFFLOAD_run): Implement using GOMP_OFFLOAD_async_run.

From-SVN: r230381

gcc/ChangeLog

@@ -1,3 +1,11 @@
+2015-11-14  Jakub Jelinek  <[email protected]>
+
+	* omp-low.c (lower_omp_ordered): Add argument to GOMP_SMD_ORDERED_*
+	internal calls - 0 if ordered simd and 1 for ordered threads simd.
+	* tree-vectorizer.c (adjust_simduid_builtins): If GOMP_SIMD_ORDERED_*
+	argument is 1, replace it with GOMP_ordered_* call instead of removing
+	it.
+
 2015-11-13  Rich Felker <[email protected]>
 
 	* config/sh/sh.md (symGOT_load): Suppress __stack_chk_guard

gcc/c/ChangeLog

@@ -1,3 +1,8 @@
+2015-11-14  Jakub Jelinek  <[email protected]>
+
+	* c-typeck.c (c_finish_omp_clauses): Don't mark
+	GOMP_MAP_FIRSTPRIVATE_POINTER decls addressable.
+
 2015-11-14  Marek Polacek  <[email protected]>
 
 	* c-decl.c: Use RECORD_OR_UNION_TYPE_P throughout.

gcc/c/c-typeck.c

@@ -12918,7 +12918,10 @@ c_finish_omp_clauses (tree clauses, bool is_omp, bool declare_simd)
 			omp_clause_code_name[OMP_CLAUSE_CODE (c)]);
 	      remove = true;
 	    }
-	  else if (!c_mark_addressable (t))
+	  else if ((OMP_CLAUSE_CODE (c) != OMP_CLAUSE_MAP
+		    || (OMP_CLAUSE_MAP_KIND (c)
+			!= GOMP_MAP_FIRSTPRIVATE_POINTER))
+		   && !c_mark_addressable (t))
 	    remove = true;
 	  else if (!(OMP_CLAUSE_CODE (c) == OMP_CLAUSE_MAP
 		     && (OMP_CLAUSE_MAP_KIND (c) == GOMP_MAP_POINTER

gcc/cp/ChangeLog

@@ -1,3 +1,8 @@
+2015-11-14  Jakub Jelinek  <[email protected]>
+
+	* semantics.c (finish_omp_clauses): Don't mark
+	GOMP_MAP_FIRSTPRIVATE_POINTER decls addressable.
+
 2015-11-13  Kai Tietz  <[email protected]>
 	    Marek Polacek  <[email protected]>
 	    Jason Merrill  <[email protected]>

gcc/cp/semantics.c

@@ -6581,6 +6581,9 @@ finish_omp_clauses (tree clauses, bool allow_fields, bool declare_simd)
 	    }
 	  else if (!processing_template_decl
 		   && TREE_CODE (TREE_TYPE (t)) != REFERENCE_TYPE
+		   && (OMP_CLAUSE_CODE (c) != OMP_CLAUSE_MAP
+		       || (OMP_CLAUSE_MAP_KIND (c)
+			   != GOMP_MAP_FIRSTPRIVATE_POINTER))
 		   && !cxx_mark_addressable (t))
 	    remove = true;
 	  else if (!(OMP_CLAUSE_CODE (c) == OMP_CLAUSE_MAP

gcc/omp-low.c

@@ -13951,8 +13951,10 @@ lower_omp_ordered (gimple_stmt_iterator *gsi_p, omp_context *ctx)
   gomp_ordered *ord_stmt = as_a <gomp_ordered *> (stmt);
   gcall *x;
   gbind *bind;
-  bool simd
-    = find_omp_clause (gimple_omp_ordered_clauses (ord_stmt), OMP_CLAUSE_SIMD);
+  bool simd = find_omp_clause (gimple_omp_ordered_clauses (ord_stmt),
+			       OMP_CLAUSE_SIMD);
+  bool threads = find_omp_clause (gimple_omp_ordered_clauses (ord_stmt),
+				  OMP_CLAUSE_THREADS);
 
   if (find_omp_clause (gimple_omp_ordered_clauses (ord_stmt),
 		       OMP_CLAUSE_DEPEND))
@@ -13975,7 +13977,8 @@ lower_omp_ordered (gimple_stmt_iterator *gsi_p, omp_context *ctx)
 
   if (simd)
     {
-      x = gimple_build_call_internal (IFN_GOMP_SIMD_ORDERED_START, 0);
+      x = gimple_build_call_internal (IFN_GOMP_SIMD_ORDERED_START, 1,
+				      build_int_cst (NULL_TREE, threads));
       cfun->has_simduid_loops = true;
     }
   else
@@ -13989,7 +13992,8 @@ lower_omp_ordered (gimple_stmt_iterator *gsi_p, omp_context *ctx)
   gimple_omp_set_body (stmt, NULL);
 
   if (simd)
-    x = gimple_build_call_internal (IFN_GOMP_SIMD_ORDERED_END, 0);
+    x = gimple_build_call_internal (IFN_GOMP_SIMD_ORDERED_END, 1,
+				    build_int_cst (NULL_TREE, threads));
   else
     x = gimple_build_call (builtin_decl_explicit (BUILT_IN_GOMP_ORDERED_END),
 			   0);

gcc/tree-vectorizer.c

@@ -177,6 +177,21 @@ adjust_simduid_builtins (hash_table<simduid_to_vf> *htab)
 	      break;
 	    case IFN_GOMP_SIMD_ORDERED_START:
 	    case IFN_GOMP_SIMD_ORDERED_END:
+	      if (integer_onep (gimple_call_arg (stmt, 0)))
+		{
+		  enum built_in_function bcode
+		    = (ifn == IFN_GOMP_SIMD_ORDERED_START
+		       ? BUILT_IN_GOMP_ORDERED_START
+		       : BUILT_IN_GOMP_ORDERED_END);
+		  gimple *g
+		    = gimple_build_call (builtin_decl_explicit (bcode), 0);
+		  tree vdef = gimple_vdef (stmt);
+		  gimple_set_vdef (g, vdef);
+		  SSA_NAME_DEF_STMT (vdef) = g;
+		  gimple_set_vuse (g, gimple_vuse (stmt));
+		  gsi_replace (&i, g, true);
+		  continue;
+		}
 	      gsi_remove (&i, true);
 	      unlink_stmt_vdef (stmt);
 	      continue;

libgomp/ChangeLog

@@ -1,3 +1,104 @@
+2015-11-14  Jakub Jelinek  <[email protected]>
+	    Aldy Hernandez  <[email protected]>
+	    Ilya Verbin  <[email protected]>
+
+	* ordered.c (gomp_doacross_init, GOMP_doacross_post,
+	GOMP_doacross_wait, gomp_doacross_ull_init, GOMP_doacross_ull_post,
+	GOMP_doacross_ull_wait): For GFS_GUIDED don't divide number of
+	iterators or IV by chunk size.
+	* parallel.c (gomp_resolve_num_threads): Don't assume that
+	if thr->ts.team is non-NULL, then pool must be non-NULL.
+	* libgomp-plugin.h (GOMP_PLUGIN_target_task_completion): Declare.
+	* libgomp.map (GOMP_PLUGIN_1.1): New symbol version, export
+	GOMP_PLUGIN_target_task_completion.
+	* Makefile.am (libgomp_la_SOURCES): Add priority_queue.c.
+	* Makefile.in: Regenerate.
+	* libgomp.h: Shuffle prototypes and forward definitions around so
+	priority queues can be defined.
+	(enum gomp_task_kind): Add GOMP_TASK_ASYNC_RUNNING.
+	(enum gomp_target_task_state): New enum.
+	(struct gomp_target_task): Add state, tgt, task and team fields.
+	(gomp_create_target_task): Change return type to bool, add
+	state argument.
+	(gomp_target_task_fn): Change return type to bool.
+	(struct gomp_device_descr): Add async_run_func.
+	(struct gomp_task): Remove children, next_child, prev_child,
+	next_queue, prev_queue, next_taskgroup, prev_taskgroup.
+	Add pnode field.
+	(struct gomp_taskgroup): Remove children.
+	Add taskgroup_queue.
+	(struct gomp_team): Change task_queue type to a priority queue.
+	(splay_compare): Define inline.
+	(priority_queue_offset): New.
+	(priority_node_to_task): New.
+	(task_to_priority_node): New.
+	* oacc-mem.c: Do not include splay-tree.h.
+	* priority_queue.c: New file.
+	* priority_queue.h: New file.
+	* splay-tree.c: Do not include splay-tree.h.
+	(splay_tree_foreach_internal): New.
+	(splay_tree_foreach): New.
+	* splay-tree.h: Become re-entrant if splay_tree_prefix is defined.
+	(splay_tree_callback): Define typedef.
+	* target.c (splay_compare): Move to libgomp.h.
+	(GOMP_target): Don't adjust *thr in any way around running offloaded
+	task.
+	(GOMP_target_ext): Likewise.  Handle target nowait.
+	(GOMP_target_update_ext, GOMP_target_enter_exit_data): Check
+	return value from gomp_create_target_task, if false, fallthrough
+	as if no dependencies exist.
+	(gomp_target_task_fn): Change return type to bool, return true
+	if the task should have another part scheduled later.  Handle
+	target nowait.
+	(gomp_load_plugin_for_device): Initialize async_run.
+	* task.c (gomp_init_task): Initialize children_queue.
+	(gomp_clear_parent_in_list): New.
+	(gomp_clear_parent_in_tree): New.
+	(gomp_clear_parent): Handle priorities.
+	(GOMP_task): Likewise.
+	(priority_queue_move_task_first,
+	gomp_target_task_completion, GOMP_PLUGIN_target_task_completion):
+	New functions.
+	(gomp_create_target_task): Use priority queues.  Change return type
+	to bool, add state argument, return false if for async
+	{{enter,exit} data,update} constructs no dependencies need to be
+	waited for, handle target nowait.  Set task->fn to NULL instead of
+	gomp_target_task_fn.
+	(verify_children_queue): Remove.
+	(priority_list_upgrade_task): New.
+	(priority_queue_upgrade_task): New.
+	(verify_task_queue): Remove.
+	(priority_list_downgrade_task): New.
+	(priority_queue_downgrade_task): New.
+	(gomp_task_run_pre): Use priority queues.
+	Abstract code out to priority_queue_downgrade_task.
+	(gomp_task_run_post_handle_dependers): Use priority queues.
+	(gomp_task_run_post_remove_parent): Likewise.
+	(gomp_task_run_post_remove_taskgroup): Likewise.
+	(gomp_barrier_handle_tasks): Likewise.  Handle target nowait target
+	tasks specially.
+	(GOMP_taskwait): Likewise.
+	(gomp_task_maybe_wait_for_dependencies): Likewise.  Abstract code to
+	priority-queue_upgrade_task.
+	(GOMP_taskgroup_start): Use priority queues.
+	(GOMP_taskgroup_end): Likewise.  Handle target nowait target tasks
+	specially.  If taskgroup is NULL, and thr->ts.level is 0, act as a
+	barrier.
+	* taskloop.c (GOMP_taskloop): Handle priorities.
+	* team.c (gomp_new_team): Call priority_queue_init.
+	(free_team): Call priority_queue_free.
+	(gomp_free_thread): Call gomp_team_end if thr->ts.team is artificial
+	team created for target nowait in implicit parallel region.
+	(gomp_team_start): For nested check, test thr->ts.level instead of
+	thr->ts.team != NULL.
+	* testsuite/libgomp.c/doacross-3.c: New test.
+	* testsuite/libgomp.c/ordered-5.c: New test.
+	* testsuite/libgomp.c/priority.c: New test.
+	* testsuite/libgomp.c/target-31.c: New test.
+	* testsuite/libgomp.c/target-32.c: New test.
+	* testsuite/libgomp.c/target-33.c: New test.
+	* testsuite/libgomp.c/target-34.c: New test.
+
 2015-11-13  Nathan Sidwell  <[email protected]>
 
 	* testsuite/libgomp.oacc-c-c++-common/loop-auto-1.c: New.

libgomp/Makefile.am

@@ -63,7 +63,7 @@ libgomp_la_SOURCES = alloc.c barrier.c critical.c env.c error.c iter.c \
 	task.c team.c work.c lock.c mutex.c proc.c sem.c bar.c ptrlock.c \
 	time.c fortran.c affinity.c target.c splay-tree.c libgomp-plugin.c \
 	oacc-parallel.c oacc-host.c oacc-init.c oacc-mem.c oacc-async.c \
-	oacc-plugin.c oacc-cuda.c
+	oacc-plugin.c oacc-cuda.c priority_queue.c
 
 include $(top_srcdir)/plugin/Makefrag.am
 

libgomp/Makefile.in

@@ -168,7 +168,7 @@ am_libgomp_la_OBJECTS = alloc.lo barrier.lo critical.lo env.lo \
 	fortran.lo affinity.lo target.lo splay-tree.lo \
 	libgomp-plugin.lo oacc-parallel.lo oacc-host.lo oacc-init.lo \
 	oacc-mem.lo oacc-async.lo oacc-plugin.lo oacc-cuda.lo \
-	$(am__objects_1)
+	priority_queue.lo $(am__objects_1)
 libgomp_la_OBJECTS = $(am_libgomp_la_OBJECTS)
 DEFAULT_INCLUDES = -I.@am__isrc@
 depcomp = $(SHELL) $(top_srcdir)/../depcomp
@@ -415,7 +415,7 @@ libgomp_la_SOURCES = alloc.c barrier.c critical.c env.c error.c iter.c \
 	bar.c ptrlock.c time.c fortran.c affinity.c target.c \
 	splay-tree.c libgomp-plugin.c oacc-parallel.c oacc-host.c \
 	oacc-init.c oacc-mem.c oacc-async.c oacc-plugin.c oacc-cuda.c \
-	$(am__append_2)
+	priority_queue.c $(am__append_2)
 
 # Nvidia PTX OpenACC plugin.
 @PLUGIN_NVPTX_TRUE@libgomp_plugin_nvptx_version_info = -version-info $(libtool_VERSION)
@@ -589,6 +589,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/oacc-plugin.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ordered.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/parallel.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/priority_queue.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/proc.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ptrlock.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/sections.Plo@am__quote@

libgomp/libgomp-plugin.h

@@ -63,6 +63,7 @@ struct addr_pair
 extern void *GOMP_PLUGIN_malloc (size_t) __attribute__ ((malloc));
 extern void *GOMP_PLUGIN_malloc_cleared (size_t) __attribute__ ((malloc));
 extern void *GOMP_PLUGIN_realloc (void *, size_t);
+void GOMP_PLUGIN_target_task_completion (void *);
 
 extern void GOMP_PLUGIN_debug (int, const char *, ...)
 	__attribute__ ((format (printf, 2, 3)));