Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/metaproject #86

Draft
wants to merge 80 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
3b3e206
Improved IFDS analysis, including demos and tests; by Mario Trageser
errt Mar 5, 2021
eb91c75
Merge branch 'develop' into feature/IDFS-improvements
errt May 27, 2021
31c4c6b
Create new project "ll"
buenaventure Jun 29, 2021
a26ee83
Add llvm bindings dependency
buenaventure Jun 29, 2021
9490098
Merge branch 'develop' into feature/IDFS-improvements
TorunR Jun 30, 2021
1a3b939
Read and dump module
buenaventure Jul 5, 2021
34d4e25
Begin implementing wrappers
buenaventure Jul 6, 2021
53da4de
Read instructions
buenaventure Jul 7, 2021
283ef68
AndroidEntryPointFinder added
Diarilex Jul 14, 2021
c88385a
Implement SimplePurityAnalysis which assumes everything to be Impure
buenaventure Nov 8, 2021
bca47d7
Merge branch 'develop' into feature/IDFS-improvements
TorunR Nov 10, 2021
c87f9ac
Implement SimplePurityAnalysis
buenaventure Nov 12, 2021
df11561
Implement viewCFG
buenaventure Nov 12, 2021
75f01ea
Merge pull request #3 from buenaventure/develop
buenaventure Dec 2, 2021
79cf52a
Merge remote-tracking branch 'upstream/develop' into ll
buenaventure Jan 5, 2022
905ad98
Add simple multilingual test project
buenaventure Jan 6, 2022
c624d84
Merge remote-tracking branch 'upstream/develop' into feature/IDFS-imp…
buenaventure Feb 17, 2022
bb079fd
Merge branch 'develop' into ll
buenaventure Feb 17, 2022
9ccee79
Add taint testcase
buenaventure Feb 5, 2022
6ab61f6
Merge branch 'feature/IDFS-improvements' into ll_multiling_ifds
buenaventure Feb 18, 2022
b99997c
Format code
buenaventure Feb 18, 2022
90c179a
Adapt new ifds to new call graph (with context)
buenaventure Feb 21, 2022
a464309
Add multilingual taint test
buenaventure Feb 23, 2022
7d00973
Extract IFDSProblem(flow functions etc) from IFDSAnalysis
buenaventure Mar 2, 2022
e3838cf
Replace specialized ifds analysis classes with parameter
buenaventure Mar 2, 2022
633c377
WIP
buenaventure Mar 5, 2022
cc84ea1
WIP
buenaventure Mar 10, 2022
cb4f4f7
WIP
buenaventure Mar 10, 2022
41d949c
WIP
buenaventure Mar 11, 2022
0b40422
WIP
buenaventure Mar 16, 2022
54bb807
WIP
buenaventure Mar 16, 2022
76a1c27
WIP (compiles)
buenaventure Mar 17, 2022
2bfaeea
Initialization fix
buenaventure Mar 17, 2022
6df8fab
Implement LLVM Argument, Type(s); Better Function/Instruction represe…
buenaventure Mar 18, 2022
f92c953
Move IFDS into separate project; Refactor handling of calls outside o…
buenaventure Mar 18, 2022
3931605
Re-implement ifds solver
buenaventure Mar 24, 2022
3ee3a04
Use new implementation for native part
buenaventure Mar 24, 2022
25d0f19
Implement forward taint analysis as test for new ifds solver
buenaventure Mar 25, 2022
3283b03
Fix new forward taint analysis
buenaventure Mar 31, 2022
313fd9a
Fix scalariformat and other compile errors
buenaventure Mar 31, 2022
75be229
Refactor: Move old code to .old, remove New prefix
buenaventure Apr 1, 2022
847baa5
First native flows working
buenaventure Apr 1, 2022
7d47c14
Fix native flows
buenaventure Apr 13, 2022
5e9cfaf
Cleanup
buenaventure May 6, 2022
16a99ae
Include llvm dependency using special sbt plugin
buenaventure May 6, 2022
89c1ff7
Fix sbt assembly
buenaventure May 11, 2022
89b91ca
Reverst scalastyle change
buenaventure May 11, 2022
91bae7d
Improve test case
buenaventure Jun 8, 2022
ed4ee71
Fix test
buenaventure Jun 9, 2022
626cce9
Fix wrong flow by adding successor to return flow and checking for ab…
buenaventure Jun 9, 2022
f34443b
Fix return flow of native code
buenaventure Jun 9, 2022
7284093
Format
buenaventure Jun 9, 2022
ab242a9
Add native to java calls to TaintTest
buenaventure Jun 16, 2022
c53aa61
Fix crashes and old tests
buenaventure Jun 17, 2022
943d7a1
Add debugData for ifds; Add simple GetElementPointer handling
buenaventure Jun 18, 2022
cb0be34
Native to Java calls WIP
buenaventure Jun 19, 2022
d3ab854
Small fixes
buenaventure Jun 22, 2022
b6a2202
JNI call analysis WIP
buenaventure Jun 25, 2022
e5d34a1
JNI call analysis WIP
buenaventure Jun 26, 2022
ec8c05a
Merge remote-tracking branch 'upstream/develop' into llvm_xlang_ifds
buenaventure Jun 30, 2022
ad65890
Fix most errors after merge
buenaventure Jun 30, 2022
c050890
Fix remaining errors after merge
buenaventure Jul 1, 2022
655d22a
Implement VTA in new IFDS for evaluation
buenaventure Jul 2, 2022
b844920
Implement Subsuming and add to evaluation
buenaventure Jul 2, 2022
d43fb11
x-lang working in simple case
buenaventure Jul 2, 2022
630162d
Bump javacpp version to work on M1 Macs.
Oct 4, 2022
52ac95c
Merge branch 'develop' into llvm_xlang_ifds
Oct 4, 2022
9786859
Remove old stuff, which should be superseded
Oct 4, 2022
91e92fa
Readd some not so obsolete Files and adapt to new IFDS Analysis
TorunR Oct 7, 2022
2d38159
Narfi ... fix Scala3 Syntax
TorunR Oct 7, 2022
4c93757
Autoformat and another backward hiccup
TorunR Oct 7, 2022
f67b853
Fix 2 small errors in llvm analysis
TorunR Oct 7, 2022
f766abb
Fix 2 other cases of scala 3 syntax.
TorunR Oct 7, 2022
55fea42
Move IFDS to si
TorunR Oct 11, 2022
42fa220
Add a language agnostic Metaproject
TorunR Oct 26, 2022
3848594
Merge branch 'develop' into feature/metaproject
TorunR Oct 27, 2022
7800e6f
Migrate TAC & AI to MetaProject
TorunR Oct 27, 2022
942f1ef
Adapt the other sub projects
TorunR Oct 28, 2022
9399a51
Fix Validate
TorunR Oct 28, 2022
e730d83
Merge branch 'develop' into feature/metaproject
TorunR Nov 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Native to Java calls WIP
  • Loading branch information
buenaventure committed Jun 19, 2022
commit cb0be34ace50429d3dd2b21e0feed5d492aa6f5f
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ public static void main (String[] args)
demo.test_native_zero_no_flow();
demo.test_native_array_tainted_flow();
demo.test_native_array_untainted_no_flow();
/*demo.test_native_call_java_sink_flow();
demo.test_native_call_java_sink_flow();
demo.test_native_call_java_source_flow();
demo.test_native_call_java_sanitize_no_flow();*/
demo.test_native_call_java_sanitize_no_flow();
System.out.println("done");
}

Expand Down Expand Up @@ -107,7 +107,7 @@ public void test_native_array_untainted_no_flow() {
this.native_array_untainted();
}

/*public void test_native_call_java_sink_flow() {
public void test_native_call_java_sink_flow() {
System.out.println("native call java sink");
this.propagate_to_java_sink(source());
}
Expand All @@ -120,7 +120,7 @@ public void test_native_call_java_source_flow() {
public void test_native_call_java_sanitize_no_flow() {
System.out.println("native call java sanitize");
this.sink(this.propagate_java_sanitize(this.source()));
}*/
}

public int indirect_source() {
return source();
Expand Down

This file was deleted.

Binary file not shown.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Binary file not shown.

This file was deleted.

Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import org.opalj.br.analyses.Project
import org.opalj.br.fpcf.FPCFAnalysesManagerKey
import org.opalj.ifds
import org.opalj.ifds.IFDSProperty
import org.opalj.ll.fpcf.analyses.ifds.LLVMStatement
import org.opalj.ll.fpcf.analyses.ifds.{LLVMFunction, LLVMStatement}
import org.opalj.ll.fpcf.analyses.ifds.taint.{JavaForwardTaintAnalysisScheduler, NativeFact, NativeForwardTaintAnalysisScheduler, NativeNullFact}
import org.opalj.ll.llvm.value.Function
import org.opalj.log.GlobalLogContext
Expand All @@ -16,18 +16,18 @@ import org.opalj.tac.fpcf.analyses.ifds.taint.{Fact, FlowFact, NullFact}
import org.scalatest.funspec.AnyFunSpec
import org.scalatest.matchers.should.Matchers

class MultilingualForwardIFDSTaintAnalysisTests extends AnyFunSpec with Matchers {
class CrossLanguageForwardIFDSTaintAnalysisTests extends AnyFunSpec with Matchers {
describe("MultilingualForwardTaintAnalysis") {
implicit val config = BaseConfig.withValue(ifds.ConfigKeyPrefix+"debug", ConfigValueFactory.fromAnyRef(true))
val project =
Project(
new java.io.File("./DEVELOPING_OPAL/validate/src/test/resources/llvm/multilingual/taint"),
new java.io.File("./DEVELOPING_OPAL/validate/src/test/resources/llvm/cross_language/taint"),
GlobalLogContext,
config
)

project.updateProjectInformationKeyInitializationData(LLVMProjectKey)(
current ⇒ List("./DEVELOPING_OPAL/validate/src/test/resources/llvm/multilingual/taint/TaintTest.ll")
current ⇒ List("./DEVELOPING_OPAL/validate/src/test/resources/llvm/cross_language/taint/TaintTest.ll")
)
project.get(LLVMProjectKey)
project.get(RTACallGraphKey)
Expand Down Expand Up @@ -61,7 +61,7 @@ class MultilingualForwardIFDSTaintAnalysisTests extends AnyFunSpec with Matchers
}

val function: Function = project.get(LLVMProjectKey).function("Java_TaintTest_native_1array_1tainted").get
val debugData = ps((function, NativeNullFact), NativeForwardTaintAnalysisScheduler.property.key).ub.asInstanceOf[IFDSProperty[LLVMStatement, NativeFact]].debugData
val debugData = ps((LLVMFunction(function), NativeNullFact), NativeForwardTaintAnalysisScheduler.property.key).ub.asInstanceOf[IFDSProperty[LLVMStatement, NativeFact]].debugData
for {
bb ← function.basicBlocks
instruction ← bb.instructions
Expand Down
2 changes: 1 addition & 1 deletion OPAL/ifds/src/main/scala/org/opalj/ifds/ICFG.scala
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package org.opalj.ifds

import scala.collection.{Set ⇒ SomeSet}

abstract class ICFG[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C, _]] {
abstract class ICFG[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[_ <: C, _]] {
/**
* Determines the statements at which the analysis starts.
*
Expand Down
8 changes: 4 additions & 4 deletions OPAL/ifds/src/main/scala/org/opalj/ifds/IFDSAnalysis.scala
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ object Dependees {
* that is the fact reaches the statement as an input.
* Source fact is the fact within the analysis entity.
*/
case class PathEdges[IFDSFact <: AbstractIFDSFact, S <: Statement[C, _], C]() {
case class PathEdges[IFDSFact <: AbstractIFDSFact, S <: Statement[_ <: C, _], C]() {
var edges = Map.empty[S, Either[Set[IFDSFact], Map[S, Set[IFDSFact]]]]

/**
Expand Down Expand Up @@ -115,7 +115,7 @@ case class PathEdges[IFDSFact <: AbstractIFDSFact, S <: Statement[C, _], C]() {
* @param source The callable and input fact for which the callable is analyzed.
* @param endSummaries Output facts of the analyzed callable as pairs of exit statement and fact
*/
protected class IFDSState[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C, _], Work](
protected class IFDSState[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[_ <: C, _], Work](
val source: (C, IFDSFact),
val dependees: Dependees[Work] = Dependees[Work](),
val pathEdges: PathEdges[IFDSFact, S, C] = PathEdges[IFDSFact, S, C](),
Expand All @@ -141,7 +141,7 @@ protected class ProjectFPCFAnalysis(val project: SomeProject) extends FPCFAnalys
* @param propertyKey Provides the concrete property key that must be unique for every distinct concrete analysis and the lower bound for the IFDSProperty.
* @tparam IFDSFact
*/
class IFDSAnalysis[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C, _]](
class IFDSAnalysis[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[_ <: C, _]](
implicit
project: SomeProject,
val ifdsProblem: IFDSProblem[IFDSFact, C, S],
Expand Down Expand Up @@ -414,7 +414,7 @@ class IFDSAnalysis[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C,
}
}

abstract class IFDSAnalysisScheduler[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C, _]]
abstract class IFDSAnalysisScheduler[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[_ <: C, _]]
extends FPCFLazyAnalysisScheduler {
final override type InitializationData = IFDSAnalysis[IFDSFact, C, S]
def property: IFDSPropertyMetaInformation[S, IFDSFact]
Expand Down
2 changes: 1 addition & 1 deletion OPAL/ifds/src/main/scala/org/opalj/ifds/IFDSProblem.scala
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import org.opalj.ifds.Dependees.Getter
* @author Mario Trageser
* @author Marc Clement
*/
abstract class IFDSProblem[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[C, _]](val icfg: ICFG[IFDSFact, C, S]) {
abstract class IFDSProblem[IFDSFact <: AbstractIFDSFact, C <: AnyRef, S <: Statement[_ <: C, _]](val icfg: ICFG[IFDSFact, C, S]) {
type Work = (S, IFDSFact, Option[S])

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ package org.opalj.ll.fpcf.analyses.ifds
import org.opalj.ifds.Callable
import org.opalj.ll.llvm.value.Function

case class LLVMFunction(val function: Function) extends Callable {
abstract class NativeFunction extends Callable {
def name: String
}

case class LLVMFunction(function: Function) extends NativeFunction {
override def name: String = function.name
override def signature: String = function.name // TODO: add signature
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,22 @@
package org.opalj.ll.fpcf.analyses.ifds

import org.opalj.ifds.{AbstractIFDSFact, ICFG}
import org.opalj.ll.llvm.value.{Call, Function, Instruction, Ret, Terminator}
import org.opalj.ll.llvm.{FunctionType, PointerType}
import org.opalj.ll.llvm.value.{Call, Function, Instruction, Ret, Terminator, Value}

class NativeForwardICFG[IFDSFact <: AbstractIFDSFact] extends ICFG[IFDSFact, Function, LLVMStatement] {
class NativeForwardICFG[IFDSFact <: AbstractIFDSFact] extends ICFG[IFDSFact, NativeFunction, LLVMStatement] {
/**
* Determines the statements at which the analysis starts.
*
* @param callable The analyzed callable.
* @return The statements at which the analysis starts.
*/
override def startStatements(callable: Function): Set[LLVMStatement] = {
if (callable.basicBlockCount == 0)
throw new IllegalArgumentException(s"${callable} does not contain any basic blocks and likely should not be in scope of the analysis")
Set(LLVMStatement(callable.entryBlock.firstInstruction))
override def startStatements(callable: NativeFunction): Set[LLVMStatement] = callable match {
case LLVMFunction(function) ⇒ {
if (function.basicBlockCount == 0)
throw new IllegalArgumentException(s"${callable} does not contain any basic blocks and likely should not be in scope of the analysis")
Set(LLVMStatement(function.entryBlock.firstInstruction))
}
}

/**
Expand All @@ -35,18 +38,27 @@ class NativeForwardICFG[IFDSFact <: AbstractIFDSFact] extends ICFG[IFDSFact, Fun
* @return All callables possibly called at the statement or None, if the statement does not
* contain a call.
*/
override def getCalleesIfCallStatement(statement: LLVMStatement): Option[collection.Set[Function]] = {
override def getCalleesIfCallStatement(statement: LLVMStatement): Option[collection.Set[NativeFunction]] = {
statement.instruction match {
case call: Call ⇒ call.calledValue match {
case function: Function ⇒ Some(Set(function))
case _ ⇒ Some(Set()) // TODO
}
case _ ⇒ None
case call: Call ⇒ Some(resolveCallee(call.calledValue))
case _ ⇒ None
}
}

override def isExitStatement(statement: LLVMStatement): Boolean = statement.instruction match {
case Ret(_) ⇒ true
case _ ⇒ false
}

private def resolveCallee(calledValue: Value): Set[NativeFunction] = calledValue match {
case function: Function ⇒ Set(LLVMFunction(function))
case _ ⇒ calledValue.typ match {
case p: PointerType ⇒ {
val functionType = p.element.asInstanceOf[FunctionType]
println(functionType.repr())
Set()
}
case _ ⇒ Set()
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,8 @@
package org.opalj.ll.fpcf.analyses.ifds

import org.opalj.br.analyses.{ProjectInformationKeys, SomeProject}
import org.opalj.ifds.{IFDSAnalysis, IFDSAnalysisScheduler, IFDSProblem, Statement}
import org.opalj.ifds.{AbstractIFDSFact, IFDSPropertyMetaInformation}
import org.opalj.ifds.{AbstractIFDSFact, IFDSAnalysis, IFDSAnalysisScheduler, IFDSProblem, IFDSPropertyMetaInformation}
import org.opalj.ll.LLVMProjectKey
import org.opalj.ll.llvm.value.{BasicBlock, Instruction}
import org.opalj.ll.llvm.value

/**
*
Expand All @@ -17,24 +14,11 @@ import org.opalj.ll.llvm.value
*/
class NativeIFDSAnalysis[IFDSFact <: AbstractIFDSFact](
project: SomeProject,
ifdsProblem: IFDSProblem[IFDSFact, value.Function, LLVMStatement],
ifdsProblem: IFDSProblem[IFDSFact, NativeFunction, LLVMStatement],
propertyKey: IFDSPropertyMetaInformation[LLVMStatement, IFDSFact]
)
extends IFDSAnalysis[IFDSFact, value.Function, LLVMStatement]()(project, ifdsProblem, propertyKey)
extends IFDSAnalysis[IFDSFact, NativeFunction, LLVMStatement]()(project, ifdsProblem, propertyKey)

/**
* A statement that is passed to the concrete analysis.
*
* @param instruction The LLVM instruction.
*/
case class LLVMStatement(instruction: Instruction) extends Statement[value.Function, BasicBlock] {
def function: value.Function = instruction.function
def basicBlock: BasicBlock = instruction.parent
override def node: BasicBlock = basicBlock
override def callable: value.Function = function
override def toString: String = s"${function.name}\n\t${instruction}\n\t${function}"
}

abstract class NativeIFDSAnalysisScheduler[IFDSFact <: AbstractIFDSFact] extends IFDSAnalysisScheduler[IFDSFact, value.Function, LLVMStatement] {
abstract class NativeIFDSAnalysisScheduler[IFDSFact <: AbstractIFDSFact] extends IFDSAnalysisScheduler[IFDSFact, NativeFunction, LLVMStatement] {
override def requiredProjectInformation: ProjectInformationKeys = Seq(LLVMProjectKey)
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@ import org.opalj.fpcf.PropertyStore
import org.opalj.ifds.Dependees.Getter
import org.opalj.ifds.{AbstractIFDSFact, IFDSProblem}
import org.opalj.ll.LLVMProjectKey
import org.opalj.ll.llvm.value.Function

abstract class NativeIFDSProblem[Fact <: AbstractIFDSFact](project: SomeProject) extends IFDSProblem[Fact, Function, LLVMStatement](new NativeForwardICFG[Fact]) {
abstract class NativeIFDSProblem[Fact <: AbstractIFDSFact](project: SomeProject) extends IFDSProblem[Fact, NativeFunction, LLVMStatement](new NativeForwardICFG[Fact]) {
final implicit val propertyStore: PropertyStore = project.get(PropertyStoreKey)
val llvmProject = project.get(LLVMProjectKey)

override def outsideAnalysisContext(callee: Function): Option[(LLVMStatement, LLVMStatement, Fact, Getter) ⇒ Set[Fact]] = {
callee.basicBlockCount match {
case 0 ⇒ Some((_: LLVMStatement, _: LLVMStatement, in: Fact, _: Getter) ⇒ Set(in))
case _ ⇒ None
}
override def outsideAnalysisContext(callee: NativeFunction): Option[(LLVMStatement, LLVMStatement, Fact, Getter) ⇒ Set[Fact]] = callee match {
case LLVMFunction(function) ⇒
function.basicBlockCount match {
case 0 ⇒ Some((_: LLVMStatement, _: LLVMStatement, in: Fact, _: Getter) ⇒ Set(in))
case _ ⇒ None
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/* BSD 2-Clause License - see OPAL/LICENSE for details. */
package org.opalj.ll.fpcf.analyses.ifds

import org.opalj.ifds.Statement
import org.opalj.ll.llvm.value.{BasicBlock, Instruction}

/**
* A statement that is passed to the concrete analysis.
*
* @param instruction The LLVM instruction.
*/
case class LLVMStatement(instruction: Instruction) extends Statement[LLVMFunction, BasicBlock] {
def function: LLVMFunction = LLVMFunction(instruction.function)
def basicBlock: BasicBlock = instruction.parent
override def node: BasicBlock = basicBlock
override def callable: LLVMFunction = function
override def toString: String = s"${function.name}\n\t${instruction}\n\t${function}"
}
Loading