Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/metaproject #86

Draft
wants to merge 80 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
3b3e206
Improved IFDS analysis, including demos and tests; by Mario Trageser
errt Mar 5, 2021
eb91c75
Merge branch 'develop' into feature/IDFS-improvements
errt May 27, 2021
31c4c6b
Create new project "ll"
buenaventure Jun 29, 2021
a26ee83
Add llvm bindings dependency
buenaventure Jun 29, 2021
9490098
Merge branch 'develop' into feature/IDFS-improvements
TorunR Jun 30, 2021
1a3b939
Read and dump module
buenaventure Jul 5, 2021
34d4e25
Begin implementing wrappers
buenaventure Jul 6, 2021
53da4de
Read instructions
buenaventure Jul 7, 2021
283ef68
AndroidEntryPointFinder added
Diarilex Jul 14, 2021
c88385a
Implement SimplePurityAnalysis which assumes everything to be Impure
buenaventure Nov 8, 2021
bca47d7
Merge branch 'develop' into feature/IDFS-improvements
TorunR Nov 10, 2021
c87f9ac
Implement SimplePurityAnalysis
buenaventure Nov 12, 2021
df11561
Implement viewCFG
buenaventure Nov 12, 2021
75f01ea
Merge pull request #3 from buenaventure/develop
buenaventure Dec 2, 2021
79cf52a
Merge remote-tracking branch 'upstream/develop' into ll
buenaventure Jan 5, 2022
905ad98
Add simple multilingual test project
buenaventure Jan 6, 2022
c624d84
Merge remote-tracking branch 'upstream/develop' into feature/IDFS-imp…
buenaventure Feb 17, 2022
bb079fd
Merge branch 'develop' into ll
buenaventure Feb 17, 2022
9ccee79
Add taint testcase
buenaventure Feb 5, 2022
6ab61f6
Merge branch 'feature/IDFS-improvements' into ll_multiling_ifds
buenaventure Feb 18, 2022
b99997c
Format code
buenaventure Feb 18, 2022
90c179a
Adapt new ifds to new call graph (with context)
buenaventure Feb 21, 2022
a464309
Add multilingual taint test
buenaventure Feb 23, 2022
7d00973
Extract IFDSProblem(flow functions etc) from IFDSAnalysis
buenaventure Mar 2, 2022
e3838cf
Replace specialized ifds analysis classes with parameter
buenaventure Mar 2, 2022
633c377
WIP
buenaventure Mar 5, 2022
cc84ea1
WIP
buenaventure Mar 10, 2022
cb4f4f7
WIP
buenaventure Mar 10, 2022
41d949c
WIP
buenaventure Mar 11, 2022
0b40422
WIP
buenaventure Mar 16, 2022
54bb807
WIP
buenaventure Mar 16, 2022
76a1c27
WIP (compiles)
buenaventure Mar 17, 2022
2bfaeea
Initialization fix
buenaventure Mar 17, 2022
6df8fab
Implement LLVM Argument, Type(s); Better Function/Instruction represe…
buenaventure Mar 18, 2022
f92c953
Move IFDS into separate project; Refactor handling of calls outside o…
buenaventure Mar 18, 2022
3931605
Re-implement ifds solver
buenaventure Mar 24, 2022
3ee3a04
Use new implementation for native part
buenaventure Mar 24, 2022
25d0f19
Implement forward taint analysis as test for new ifds solver
buenaventure Mar 25, 2022
3283b03
Fix new forward taint analysis
buenaventure Mar 31, 2022
313fd9a
Fix scalariformat and other compile errors
buenaventure Mar 31, 2022
75be229
Refactor: Move old code to .old, remove New prefix
buenaventure Apr 1, 2022
847baa5
First native flows working
buenaventure Apr 1, 2022
7d47c14
Fix native flows
buenaventure Apr 13, 2022
5e9cfaf
Cleanup
buenaventure May 6, 2022
16a99ae
Include llvm dependency using special sbt plugin
buenaventure May 6, 2022
89c1ff7
Fix sbt assembly
buenaventure May 11, 2022
89b91ca
Reverst scalastyle change
buenaventure May 11, 2022
91bae7d
Improve test case
buenaventure Jun 8, 2022
ed4ee71
Fix test
buenaventure Jun 9, 2022
626cce9
Fix wrong flow by adding successor to return flow and checking for ab…
buenaventure Jun 9, 2022
f34443b
Fix return flow of native code
buenaventure Jun 9, 2022
7284093
Format
buenaventure Jun 9, 2022
ab242a9
Add native to java calls to TaintTest
buenaventure Jun 16, 2022
c53aa61
Fix crashes and old tests
buenaventure Jun 17, 2022
943d7a1
Add debugData for ifds; Add simple GetElementPointer handling
buenaventure Jun 18, 2022
cb0be34
Native to Java calls WIP
buenaventure Jun 19, 2022
d3ab854
Small fixes
buenaventure Jun 22, 2022
b6a2202
JNI call analysis WIP
buenaventure Jun 25, 2022
e5d34a1
JNI call analysis WIP
buenaventure Jun 26, 2022
ec8c05a
Merge remote-tracking branch 'upstream/develop' into llvm_xlang_ifds
buenaventure Jun 30, 2022
ad65890
Fix most errors after merge
buenaventure Jun 30, 2022
c050890
Fix remaining errors after merge
buenaventure Jul 1, 2022
655d22a
Implement VTA in new IFDS for evaluation
buenaventure Jul 2, 2022
b844920
Implement Subsuming and add to evaluation
buenaventure Jul 2, 2022
d43fb11
x-lang working in simple case
buenaventure Jul 2, 2022
630162d
Bump javacpp version to work on M1 Macs.
Oct 4, 2022
52ac95c
Merge branch 'develop' into llvm_xlang_ifds
Oct 4, 2022
9786859
Remove old stuff, which should be superseded
Oct 4, 2022
91e92fa
Readd some not so obsolete Files and adapt to new IFDS Analysis
TorunR Oct 7, 2022
2d38159
Narfi ... fix Scala3 Syntax
TorunR Oct 7, 2022
4c93757
Autoformat and another backward hiccup
TorunR Oct 7, 2022
f67b853
Fix 2 small errors in llvm analysis
TorunR Oct 7, 2022
f766abb
Fix 2 other cases of scala 3 syntax.
TorunR Oct 7, 2022
55fea42
Move IFDS to si
TorunR Oct 11, 2022
42fa220
Add a language agnostic Metaproject
TorunR Oct 26, 2022
3848594
Merge branch 'develop' into feature/metaproject
TorunR Oct 27, 2022
7800e6f
Migrate TAC & AI to MetaProject
TorunR Oct 27, 2022
942f1ef
Adapt the other sub projects
TorunR Oct 28, 2022
9399a51
Fix Validate
TorunR Oct 28, 2022
e730d83
Merge branch 'develop' into feature/metaproject
TorunR Nov 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Readd some not so obsolete Files and adapt to new IFDS Analysis
- Commented out Backward analyses for now.
- Moved forward class for name analysis to new IFDS Analysis
- Moved VTA Analysis Test (including its matchers) to the new IFDS Analysis
  • Loading branch information
TorunR committed Oct 7, 2022
commit 91e92fa92376a7f9fd2eff796344a0c5157a8ca9
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
/* BSD 2-Clause License - see OPAL/LICENSE for details. */
package org.opalj.tac.fpcf.analyses.taint
/* TODO Fix as soon as backwards analysis is implemented
import org.opalj.br.analyses.SomeProject
import org.opalj.br.{DeclaredMethod, DefinedMethod, Method}
import org.opalj.fpcf.{EPS, FinalEP, PropertyStore}
import org.opalj.ifds.IFDSPropertyMetaInformation
import org.opalj.tac.cg.RTACallGraphKey
import org.opalj.tac.fpcf.analyses.ifds.JavaMethod
import org.opalj.tac.fpcf.analyses.ifds.old.taint.BackwardTaintProblem
import org.opalj.tac.fpcf.analyses.ifds.old._
import org.opalj.tac.fpcf.analyses.ifds.taint._
import org.opalj.tac.fpcf.properties.OldTaint

import java.io.File

/**
* A backward IFDS taint analysis, which tracks the String parameters of all methods of the rt.jar,
* * which are callable from outside the library, to calls of Class.forName.
*
* @author Mario Trageser
*/
class BackwardClassForNameTaintAnalysisScheduler private (implicit val project: SomeProject)
extends BackwardIFDSAnalysis(new BackwardClassForNameTaintProblem(project), OldTaint)

class BackwardClassForNameTaintProblem(p: SomeProject) extends BackwardTaintProblem(p) {

/**
* The string parameters of all public methods are entry points.
*/
override val entryPoints: Seq[(DeclaredMethod, TaintFact)] =
p.allProjectClassFiles.filter(classFile =>
classFile.thisType.fqn == "java/lang/Class")
.flatMap(classFile => classFile.methods)
.filter(_.name == "forName")
.map(method => declaredMethods(method) -> Variable(-2))

/**
* There is no sanitizing in this analysis.
*/
override protected def sanitizesReturnValue(callee: DeclaredMethod): Boolean = false

/**
* There is no sanitizing in this analysis.
*/
override protected def sanitizesParameter(call: DeclaredMethodJavaStatement, in: TaintFact): Boolean = false

/**
* Do not perform unbalanced return for methods, which can be called from outside the library.
*/
override def shouldPerformUnbalancedReturn(source: (DeclaredMethod, TaintFact)): Boolean = {
super.shouldPerformUnbalancedReturn(source) &&
(!canBeCalledFromOutside(source._1) ||
// The source is callable from outside, but should create unbalanced return facts.
entryPoints.contains(source))
}

/**
* This analysis does not create FlowFacts at calls.
* Instead, FlowFacts are created at the start node of methods.
*/
override protected def createFlowFactAtCall(call: DeclaredMethodJavaStatement, in: Set[TaintFact],
source: (DeclaredMethod, TaintFact)): Option[FlowFact] = None

/**
* This analysis does not create FlowFacts at returns.
* Instead, FlowFacts are created at the start node of methods.
*/
protected def applyFlowFactFromCallee(
calleeFact: FlowFact,
source: (DeclaredMethod, TaintFact)
): Option[FlowFact] = None

/**
* If we analyzed a transitive caller of the sink, which is callable from outside the library,
* and a formal parameter is tainted, we create a FlowFact.
*/
override protected def createFlowFactAtBeginningOfMethod(
in: Set[TaintFact],
source: (DeclaredMethod, TaintFact)
): Option[FlowFact] = {
if (source._2.isInstanceOf[UnbalancedReturnFact[TaintFact @unchecked]] &&
canBeCalledFromOutside(source._1) && in.exists {
// index < 0 means, that it is a parameter.
case Variable(index) if index < 0 => true
case ArrayElement(index, _) if index < 0 => true
case InstanceField(index, _, _) if index < 0 => true
case _ => false
}) {
Some(FlowFact(currentCallChain(source).map(JavaMethod(_))))
} else None
}
}

object BackwardClassForNameTaintAnalysisScheduler extends IFDSAnalysisScheduler[TaintFact] {

override def init(p: SomeProject, ps: PropertyStore): BackwardClassForNameTaintAnalysisScheduler = {
p.get(RTACallGraphKey)
new BackwardClassForNameTaintAnalysisScheduler()(p)
}

override def property: IFDSPropertyMetaInformation[DeclaredMethodJavaStatement, TaintFact] = OldTaint
}

class BackwardClassForNameTaintAnalysisRunner extends AbsractIFDSAnalysisRunner {

override def analysisClass: BackwardClassForNameTaintAnalysisScheduler.type = BackwardClassForNameTaintAnalysisScheduler

override def printAnalysisResults(analysis: AbstractIFDSAnalysis[_], ps: PropertyStore): Unit = {
val propertyKey = BackwardClassForNameTaintAnalysisScheduler.property.key
val flowFactsAtSources = ps.entities(propertyKey).collect {
case EPS((m: DefinedMethod, inputFact)) if canBeCalledFromOutside(m, ps) =>
(m, inputFact)
}.flatMap(ps(_, propertyKey) match {
case FinalEP(_, OldTaint(result, _)) =>
result.values.fold(Set.empty)((acc, facts) => acc ++ facts).filter {
case FlowFact(_) => true
case _ => false
}
case _ => Seq.empty
})
for {
fact <- flowFactsAtSources
} {
fact match {
case FlowFact(flow) => println(s"flow: "+flow.asInstanceOf[Seq[Method]].map(_.toJava).mkString(", "))
case _ =>
}
}
}
}

object BackwardClassForNameTaintAnalysisRunner {
def main(args: Array[String]): Unit = {
if (args.contains("--help")) {
println("Potential parameters:")
println(" -seq (to use the SequentialPropertyStore)")
println(" -l2 (to use the l2 domain instead of the default l1 domain)")
println(" -delay (for a three seconds delay before the taint flow analysis is started)")
println(" -debug (for debugging mode in the property store)")
println(" -evalSchedulingStrategies (evaluates all available scheduling strategies)")
println(" -f <file> (Stores the average runtime to this file)")
} else {
val fileIndex = args.indexOf("-f")
new BackwardClassForNameTaintAnalysisRunner().run(
args.contains("-debug"),
args.contains("-l2"),
args.contains("-delay"),
args.contains("-evalSchedulingStrategies"),
if (fileIndex >= 0) Some(new File(args(fileIndex + 1))) else None
)
}
}
}*/
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
/* BSD 2-Clause License - see OPAL/LICENSE for details. */
package org.opalj.tac.fpcf.analyses.taint

import org.opalj.br.analyses.{DeclaredMethodsKey, ProjectInformationKeys, SomeProject}
import org.opalj.br.fpcf.PropertyStoreKey
import org.opalj.br.{DeclaredMethod, Method, ObjectType}
import org.opalj.fpcf.{FinalEP, PropertyBounds, PropertyStore}
import org.opalj.ifds.{IFDSAnalysis, IFDSAnalysisScheduler, IFDSProperty, IFDSPropertyMetaInformation}
import org.opalj.tac.cg.{RTACallGraphKey, TypeIteratorKey}
import org.opalj.tac.fpcf.analyses.ifds.taint.{FlowFact, ForwardTaintProblem, TaintFact, TaintProblem, Variable}
import org.opalj.tac.fpcf.analyses.ifds.*
import org.opalj.tac.fpcf.properties.{TACAI, Taint}
import org.opalj.tac.fpcf.properties.cg.Callers

import java.io.File

/**
* A forward IFDS taint analysis, which tracks the String parameters of all methods of the rt.jar,
* which are callable from outside the library, to calls of Class.forName.
*
* @author Dominik Helm
* @author Mario Trageser
* @author Michael Eichberg
*/
class ForwardClassForNameTaintAnalysis(project: SomeProject)
extends IFDSAnalysis()(project,new ForwardClassForNameTaintProblem(project),Taint)

class ForwardClassForNameTaintProblem(project: SomeProject)
extends ForwardTaintProblem(project) with TaintProblem[Method, JavaStatement, TaintFact] {
private val propertyStore = project.get(PropertyStoreKey)
/**
* Returns all methods, that can be called from outside the library.
* The call graph must be computed, before this method may be invoked.
*
* @return All methods, that can be called from outside the library.
*/
protected def methodsCallableFromOutside: Set[DeclaredMethod] = {
declaredMethods.declaredMethods.filter(canBeCalledFromOutside).toSet
}

/**
* Checks, if some `method` can be called from outside the library.
* The call graph must be computed, before this method may be invoked.
*
* @param method The method, which may be callable from outside.
* @return True, if `method` can be called from outside the library.
*/
protected def canBeCalledFromOutside(method: DeclaredMethod): Boolean = {
val FinalEP(_, callers) = propertyStore(method, Callers.key)
callers.hasCallersWithUnknownContext
}
/**
* The string parameters of all public methods are entry points.
*/
override def entryPoints: Seq[(Method, TaintFact)] = for {
m <- methodsCallableFromOutside.toSeq
if !m.definedMethod.isNative
index <- m.descriptor.parameterTypes.zipWithIndex.collect {
case (pType, index) if pType == ObjectType.String => index
}
} yield (m.definedMethod, Variable(-2 - index))

/**
* There is no sanitizing in this analysis.
*/
override protected def sanitizesReturnValue(callee: Method): Boolean = false

/**
* There is no sanitizing in this analysis.
*/
override protected def sanitizesParameter(call: JavaStatement, in: TaintFact): Boolean = false

/**
* This analysis does not create new taints on the fly.
* Instead, the string parameters of all public methods are tainted in the entry points.
*/
override protected def createTaints(callee: Method, call: JavaStatement): Set[TaintFact] =
Set.empty

/**
* Create a FlowFact, if Class.forName is called with a tainted variable for the first parameter.
*/
override protected def createFlowFact(callee: Method, call: JavaStatement,
in: TaintFact): Option[FlowFact] = {
if (isClassForName(declaredMethods(callee)) && in == Variable(-2))
Some(FlowFact(Seq(JavaMethod(call.method))))
else None
}

/**
* Checks, if a `method` is Class.forName.
*
* @param method The method.
* @return True, if the method is Class.forName.
*/
private def isClassForName(method: DeclaredMethod): Boolean =
method.declaringClassType == ObjectType.Class && method.name == "forName"
}

object ForwardClassForNameTaintAnalysisScheduler extends IFDSAnalysisScheduler[TaintFact,Method,JavaStatement] {

override def init(p: SomeProject, ps: PropertyStore) = new ForwardClassForNameTaintAnalysis(p)

override def property: IFDSPropertyMetaInformation[JavaStatement, TaintFact] = Taint

override def requiredProjectInformation: ProjectInformationKeys = Seq(DeclaredMethodsKey, TypeIteratorKey, PropertyStoreKey,RTACallGraphKey)

override def uses: Set[PropertyBounds] = Set(PropertyBounds.finalP(TACAI), PropertyBounds.finalP(Callers))
}

class ForwardClassForNameAnalysisRunner extends EvaluationRunner {

override def analysisClass: ForwardClassForNameTaintAnalysisScheduler.type = ForwardClassForNameTaintAnalysisScheduler

override def printAnalysisResults(analysis: IFDSAnalysis[?,?,?], ps: PropertyStore): Unit =
for {
e <- analysis.ifdsProblem.entryPoints
flows = ps(e, ForwardClassForNameTaintAnalysisScheduler.property.key)
fact <- flows.ub.asInstanceOf[IFDSProperty[JavaStatement, TaintFact]].flows.values.flatten.toSet[TaintFact]
} {
fact match {
case FlowFact(flow) => println(s"flow: "+flow.asInstanceOf[Set[Method]].map(_.toJava).mkString(", "))
case _ =>
}
}
}

object ForwardClassForNameAnalysisRunner {
def main(args: Array[String]): Unit = {
if (args.contains("--help")) {
println("Potential parameters:")
println(" -seq (to use the SequentialPropertyStore)")
println(" -l2 (to use the l2 domain instead of the default l1 domain)")
println(" -delay (for a three seconds delay before the taint flow analysis is started)")
println(" -debug (for debugging mode in the property store)")
println(" -evalSchedulingStrategies (evaluates all available scheduling strategies)")
println(" -f <file> (Stores the average runtime to this file)")
} else {
val fileIndex = args.indexOf("-f")
new ForwardClassForNameAnalysisRunner().run(
args.contains("-debug"),
args.contains("-l2"),
args.contains("-delay"),
args.contains("-evalSchedulingStrategies"),
if (fileIndex >= 0) Some(new File(args(fileIndex + 1))) else None
)
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/* BSD 2-Clause License - see OPAL/LICENSE for details. */
package org.opalj.fpcf.ifds

import org.opalj.ai.domain.l2
import org.opalj.ai.fpcf.properties.AIDomainFactoryKey
import org.opalj.br.analyses.Project
import org.opalj.fpcf.PropertiesTest
import org.opalj.fpcf.properties.vta.{ExpectedCallee, ExpectedType}
import org.opalj.tac.cg.RTACallGraphKey
import org.opalj.tac.fpcf.analyses.ifds.VTANullFact
import org.opalj.tac.fpcf.analyses.ifds.IFDSBasedVariableTypeAnalysisScheduler

import java.net.URL

class VTATest extends PropertiesTest {

override def init(p: Project[URL]): Unit = {
p.updateProjectInformationKeyInitializationData(
AIDomainFactoryKey
)(
(_: Option[Set[Class[_ <: AnyRef]]]) =>
Set[Class[_ <: AnyRef]](
classOf[l2.DefaultPerformInvocationsDomainWithCFGAndDefUse[URL]]
)
)
p.get(RTACallGraphKey)
}

describe("Test the ExpectedType annotations") {
val testContext = executeAnalyses(new IFDSBasedVariableTypeAnalysisScheduler)
val project = testContext.project
// val declaredMethods = project.get(DeclaredMethodsKey)
val eas = methodsWithAnnotations(project).map {
case (method, entityString, annotations) =>
((method, VTANullFact), entityString, annotations)
}
testContext.propertyStore.shutdown()
validateProperties(testContext, eas, Set(ExpectedType.PROPERTY_VALIDATOR_KEY))
validateProperties(testContext, eas, Set(ExpectedCallee.PROPERTY_VALIDATOR_KEY))
}

}
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
/* BSD 2-Clause License - see OPAL/LICENSE for details. */
package org.opalj.fpcf.properties.taint

/* TODO Fix as soon as backwards analysis is implemented
import org.opalj.br._
import org.opalj.br.analyses.SomeProject
import org.opalj.br.fpcf.PropertyStoreKey
import org.opalj.fpcf.properties.AbstractPropertyMatcher
import org.opalj.fpcf.{EPS, Entity, FinalEP, Property}
import org.opalj.tac.fpcf.analyses.ifds.taint.old.BackwardTaintAnalysisFixtureScheduler
import org.opalj.tac.fpcf.analyses.ifds.taint.BackwardTaintAnalysisFixtureScheduler
import org.opalj.tac.fpcf.analyses.ifds.taint.{TaintFact, FlowFact}
import org.opalj.tac.fpcf.properties.OldTaint

Expand Down Expand Up @@ -48,3 +49,4 @@ class BackwardFlowPathMatcher extends AbstractPropertyMatcher {
}
}
}
*/
Loading