A Case for Compiler-driven Superpage Allocation

", Context); > + ? ?std::string Error; > + ? ?TheJIT.reset(EngineBuilder(M).setEngineKind(EngineKind::JIT) > + ? ? ? ? ? ? ? ? .setErrorStr(&Error).create()); > + ? ?ASSERT_TRUE(TheJIT.get() != NULL) << Error; > + ?} > + > + ?LLVMContext Context; > + ?Module *M; ?// Owned by ExecutionEngine. > + ?OwningPtr TheJIT; > +}; > + > ?// Regression test for a bug. ?The JIT used to allocate globals inside the same > ?// memory block used for the function, and when the function code was freed, > ?// the global was left in the same place. ?This test allocates a function > @@ -115,6 +131,43 @@ > ? EXPECT_EQ(3, *GPtr); > ?} > > +int PlusOne(int arg) { > + ?return arg + 1; > +} > + > +TEST_F(JITTest, FarCallToKnownFunction) { > + ?// x86-64 can only make direct calls to functions within 32 bits of > + ?// the current PC. ?To call anything farther away, we have to load > + ?// the address into a register and call through the register. ?The > + ?// current JIT does this by allocating a stub for any far call. > + ?// There was a bug in which the JIT tried to emit a direct call when > + ?// the target was already in the JIT's global mappings and lazy > + ?// compilation was disabled. > + > + ?Function *KnownFunction = Function::Create( > + ? ? ?TypeBuilder::get(Context), > + ? ? ?GlobalValue::ExternalLinkage, "known", M); > + ?TheJIT->addGlobalMapping(KnownFunction, (void*)(intptr_t)PlusOne); > + > + ?// int test() { return known(7); } > + ?Function *TestFunction = Function::Create( > + ? ? ?TypeBuilder::get(Context), > + ? ? ?GlobalValue::ExternalLinkage, "test", M); > + ?BasicBlock *Entry = BasicBlock::Create(Context, "entry", TestFunction); > + ?IRBuilder<> Builder(Entry); > + ?Value *result = Builder.CreateCall( > + ? ? ?KnownFunction, > + ? ? ?ConstantInt::get(TypeBuilder::get(Context), 7)); > + ?Builder.CreateRet(result); > + > + ?TheJIT->EnableDlsymStubs(false); > + ?TheJIT->DisableLazyCompilation(); > + ?int (*TestFunctionPtr)() = reinterpret_cast( > + ? ? ?(intptr_t)TheJIT->getPointerToFunction(TestFunction)); > + ?// This used to crash in trying to call PlusOne(). > + ?EXPECT_EQ(8, TestFunctionPtr()); > +} > + > ?// This code is copied from JITEventListenerTest, but it only runs once for all > ?// the tests in this directory. ?Everything seems fine, but that's strange > ?// behavior. > > > _______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits > From sabre at nondot.org Mon Oct 26 01:20:24 2009 From: sabre at nondot.org (Chris Lattner) Date: Mon, 26 Oct 2009 01:20:24 -0500 Subject: [llvm-commits] CVS: llvm-www/pubs/2009-08-Zoltar.html 2009-08-Zoltar.pdf pubs.js Message-ID: <200910260620.n9Q6KOWW006712@zion.cs.uiuc.edu> Changes in directory llvm-www/pubs: 2009-08-Zoltar.html added (r1.1) 2009-08-Zoltar.pdf added (r1.1) pubs.js updated: 1.67 -> 1.68 --- Log message: add Zoltar: a spectrum-based fault localization tool and another one without a pdf. --- Diffs of the changes: (+79 -0) 2009-08-Zoltar.html | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2009-08-Zoltar.pdf | 0 pubs.js | 15 ++++++++++++ 3 files changed, 79 insertions(+) Index: llvm-www/pubs/2009-08-Zoltar.html diff -c /dev/null llvm-www/pubs/2009-08-Zoltar.html:1.1 *** /dev/null Mon Oct 26 01:19:55 2009 --- llvm-www/pubs/2009-08-Zoltar.html Mon Oct 26 01:19:45 2009 *************** *** 0 **** --- 1,64 ---- + + + + + + Zoltar: a spectrum-based fault localization tool + + + +

+ Zoltar: a spectrum-based fault localization tool +

+ Tom Janssen, Rui Abreu, and Arjan J.C. van Gemund +

+ +

Abstract:

+ Locating software components which are responsible for observed failures is the most expensive, error-prone phase in the software development life cycle. Automated diagnosis of software faults can improve the efficiency of the debugging process, and is therefore an important process for the development of dependable software. In this paper we present a toolset for automatic fault localization, dubbed Zoltar, which adopts a spectrum-based fault localization technique. The toolset provides the infrastructure to automatically instrument the source code of software programs to produce runtime data, which is subsequently analyzed to return a ranked list of likely faulty locations. Aimed at total automation (e.g., for runtime fault diagnosis), Zoltar has the capability of instrumenting the program under analysis with fault screeners, for automatic error detection. Using a small thread-based example program as well as a large realistic program, we show the applicability of the pr! oposed toolset. +

+ +

Published:

+ "Zoltar: a spectrum-based fault localization tool" +
+ Tom Janssen, Rui Abreu, and Arjan J.C. van Gemund. +
+ + Proceedings of the 2009 ESEC/FSE workshop on Software integration and evolution @ runtime + , Amsterdam, The Netherlands, August 2009. +

Download:

Paper:

+ Zoltar: a spectrum-based fault localization tool + (PDF)

+ +

BibTeX Entry:

+ @inproceedings{1596502,
+  author = {Janssen, Tom and Abreu, Rui and van Gemund, Arjan J.C.},
+  title = {Zoltar: a spectrum-based fault localization tool},
+  booktitle = {SINTER '09: Proceedings of the 2009 ESEC/FSE workshop on Software integration and evolution @ runtime},
+  year = {2009},
+  isbn = {978-1-60558-681-6},
+  pages = {23--30},
+  location = {Amsterdam, The Netherlands},
+  doi = {http://doi.acm.org/10.1145/1596495.1596502},
+  publisher = {ACM},
+  address = {New York, NY, USA},
+  }
+

+ + +

+ + + Index: llvm-www/pubs/2009-08-Zoltar.pdf Index: llvm-www/pubs/pubs.js diff -u llvm-www/pubs/pubs.js:1.67 llvm-www/pubs/pubs.js:1.68 --- llvm-www/pubs/pubs.js:1.67 Mon Oct 26 01:03:17 2009 +++ llvm-www/pubs/pubs.js Mon Oct 26 01:19:45 2009 @@ -33,6 +33,14 @@ month: 8, year: 2009}, + {url: '2009-08-Zoltar.html', + title: "Zoltar: a spectrum-based fault localization tool", + author: "Tom Janssen, Rui Abreu, and Arjan J.C. van Gemund", + published: "Proc. of the 2009 ESEC/FSE workshop on Software integration and evolution @ runtime", + location: "Amsterdam, The Netherlands", + month: 8, + year: 2009}, + {url: '2009-07-ISSTA-BegBunch.html', title: 'BegBunch: benchmarking for C bug detection tools', author: 'Cristina Cifuentes, Christian Hoermann, Nathan Keynes, Lian Li, Simon Long, Erica Mealy, Michael Mounteney, and Bernhard Scholz', @@ -172,6 +180,13 @@ month: 3, year: 2009}, + {url: "", + title: "SORU: A Reconfigurable Vector Unit for Adaptable Embedded Systems", + author: "Jos?? M. Moya, Javier Rodr??guez, et al", + published: "Proc. of the 5th International Workshop on Reconfigurable Computing: Architectures, Tools and Applications", + month: 3, + year: 2009}, + {url: '2009-02-PPoPP-MappingParallelism.html', title: 'Mapping parallelism to multi-cores: a machine learning based approach', author: "Zheng Wang and Michael F.P. O'Boyle", From sabre at nondot.org Mon Oct 26 01:29:41 2009 From: sabre at nondot.org (Chris Lattner) Date: Mon, 26 Oct 2009 01:29:41 -0500 Subject: [llvm-commits] CVS: llvm-www/pubs/2009-08-FSE-Altair.html 2009-08-FSE-Altair.pdf pubs.js Message-ID: <200910260629.n9Q6Tf4P007045@zion.cs.uiuc.edu> Changes in directory llvm-www/pubs: 2009-08-FSE-Altair.html added (r1.1) 2009-08-FSE-Altair.pdf added (r1.1) pubs.js updated: 1.68 -> 1.69 --- Log message: add a couple more. --- Diffs of the changes: (+70 -0) 2009-08-FSE-Altair.html | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 2009-08-FSE-Altair.pdf | 0 pubs.js | 15 +++++++++++++ 3 files changed, 70 insertions(+) Index: llvm-www/pubs/2009-08-FSE-Altair.html diff -c /dev/null llvm-www/pubs/2009-08-FSE-Altair.html:1.1 *** /dev/null Mon Oct 26 01:29:34 2009 --- llvm-www/pubs/2009-08-FSE-Altair.html Mon Oct 26 01:29:24 2009 *************** *** 0 **** --- 1,55 ---- + + + + + + API hyperlinking via structural overlap + + + +

+ API hyperlinking via structural overlap +

+ Fan Long, Xi Wang, and Yang Cai +

+ +

Abstract:

+ This paper presents a tool Altair that automatically generates API function cross-references, which emphasizes reliable structural measures and does not depend on specific client code. Altair ranks related API functions for a given query according to pair-wise overlap, i.e., how they share state, and clusters tightly related ones into meaningful modules.
+ + Experiments against several popular C software packages show that Altair recommends related API functions for a given query with remarkably more precise and complete results than previous tools, that it can extract modules from moderate-sized software (e.g., Apache with 1000+ functions) at high precision and recall rates (e.g., both exceeding 70% for two modules in Apache), and that the computation can finish within a few seconds. +

+ +

Published:

+ "API hyperlinking via structural overlap" +
+ Fan Long, Xi Wang, and Yang Cai. +
+ + Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering on European software engineering conference and foundations of software engineering symposium (FSE'09) + , Amsterdam, The Netherlands, August 2009. +

Download:

Paper:

+ API hyperlinking via structural overlap + (PDF)

+ +

BibTeX Entry:

+ BIBTEX
+

+ + +

+ + + Index: llvm-www/pubs/2009-08-FSE-Altair.pdf Index: llvm-www/pubs/pubs.js diff -u llvm-www/pubs/pubs.js:1.68 llvm-www/pubs/pubs.js:1.69 --- llvm-www/pubs/pubs.js:1.68 Mon Oct 26 01:19:45 2009 +++ llvm-www/pubs/pubs.js Mon Oct 26 01:29:24 2009 @@ -17,6 +17,14 @@ month: 9, year: 2009}, + {url: "2009-08-FSE-Altair.html", + title: "API hyperlinking via structural overlap", + published: "Proc. of the ACM symposium on The foundations of software engineering symposium (FSE'09)", + author: "Fan Long, Xi Wang, and Yang Cai", + location: "Amsterdam, The Netherlands", + month: 8, + year: 2009}, + {url: '2009-08-SAS-IPSSA.html', title: "Increasing the scope and resolution of Interprocedural Static Single Assignment", author: "Silvian Calman and Jianwen Zhu", @@ -187,6 +195,13 @@ month: 3, year: 2009}, + {url: "", + title: "Compiling Techniques for Coarse Grained Runtime Reconfigurable Architecture", + author: "Mythri Alle, Keshavan Varadarajan, Alexander Fell, S. K. Nandy and Ranjani Narayan", + published: "Proc. of the 5th International Workshop on Reconfigurable Computing: Architectures, Tools and Applications", + month: 3, + year: 2009}, + {url: '2009-02-PPoPP-MappingParallelism.html', title: 'Mapping parallelism to multi-cores: a machine learning based approach', author: "Zheng Wang and Michael F.P. O'Boyle", From jyasskin at google.com Mon Oct 26 02:19:16 2009 From: jyasskin at google.com (Jeffrey Yasskin) Date: Mon, 26 Oct 2009 00:19:16 -0700 Subject: [llvm-commits] [llvm] r83353 - in /llvm/trunk: lib/ExecutionEngine/JIT/JITEmitter.cpp unittests/ExecutionEngine/JIT/JITTest.cpp In-Reply-To: <6a8523d60910252308p3db171bh75a1ce1d6669726b@mail.gmail.com> References: <200910060035.n960Zuca025893@zion.cs.uiuc.edu> <6a8523d60910252308p3db171bh75a1ce1d6669726b@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 11:08 PM, Daniel Dunbar wrote: > On Mon, Oct 5, 2009 at 5:35 PM, Jeffrey Yasskin wrote: >> Author: jyasskin >> Date: Mon Oct ?5 19:35:55 2009 >> New Revision: 83353 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=83353&view=rev >> Log: >> Fix http://llvm.org/PR5116 by rolling back r60822. ?This passes `make unittests >> check-lit` on both x86-64 Linux and x86-32 Darwin. > > Note that the unittests are automatically run as part of 'make check-lit'. Thanks :) From gohman at apple.com Mon Oct 26 10:32:57 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 15:32:57 -0000 Subject: [llvm-commits] [llvm] r85118 - in /llvm/trunk: lib/Transforms/Scalar/LoopStrengthReduce.cpp test/CodeGen/X86/negative-stride-fptosi-user.ll Message-ID: <200910261532.n9QFWvqE018728@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 10:32:57 2009 New Revision: 85118 URL: http://llvm.org/viewvc/llvm-project?rev=85118&view=rev Log: Make LSR's OptimizeShadowIV ignore induction variables with negative strides for now, because it doesn't handle them correctly. This fixes a miscompile of SingleSource/Benchmarks/Misc-C++/ray. This problem was usually hidden because indvars transforms such induction variables into negations of canonical induction variables. Added: llvm/trunk/test/CodeGen/X86/negative-stride-fptosi-user.ll Modified: llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp Modified: llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp?rev=85118&r1=85117&r2=85118&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp (original) +++ llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp Mon Oct 26 10:32:57 2009 @@ -2262,6 +2262,10 @@ if (!C) continue; + // Ignore negative constants, as the code below doesn't handle them + // correctly. TODO: Remove this restriction. + if (!C->getValue().isStrictlyPositive()) continue; + /* Add new PHINode. */ PHINode *NewPH = PHINode::Create(DestTy, "IV.S.", PH); Added: llvm/trunk/test/CodeGen/X86/negative-stride-fptosi-user.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/negative-stride-fptosi-user.ll?rev=85118&view=auto ============================================================================== --- llvm/trunk/test/CodeGen/X86/negative-stride-fptosi-user.ll (added) +++ llvm/trunk/test/CodeGen/X86/negative-stride-fptosi-user.ll Mon Oct 26 10:32:57 2009 @@ -0,0 +1,25 @@ +; RUN: llc < %s -march=x86-64 | grep cvtsi2sd + +; LSR previously eliminated the sitofp by introducing an induction +; variable which stepped by a bogus ((double)UINT32_C(-1)). It's theoretically +; possible to eliminate the sitofp using a proper -1.0 step though; this +; test should be changed if that is done. + +define void @foo(i32 %N) nounwind { +entry: + %0 = icmp slt i32 %N, 0 ; [#uses=1] + br i1 %0, label %bb, label %return + +bb: ; preds = %bb, %entry + %i.03 = phi i32 [ 0, %entry ], [ %2, %bb ] ; [#uses=2] + %1 = sitofp i32 %i.03 to double ; [#uses=1] + tail call void @bar(double %1) nounwind + %2 = add nsw i32 %i.03, -1 ; [#uses=2] + %exitcond = icmp eq i32 %2, %N ; [#uses=1] + br i1 %exitcond, label %return, label %bb + +return: ; preds = %bb, %entry + ret void +} + +declare void @bar(double) From sabre at nondot.org Mon Oct 26 10:40:08 2009 From: sabre at nondot.org (Chris Lattner) Date: Mon, 26 Oct 2009 15:40:08 -0000 Subject: [llvm-commits] [llvm] r85119 - in /llvm/trunk: lib/Transforms/Scalar/InstructionCombining.cpp test/Transforms/InstCombine/or.ll Message-ID: <200910261540.n9QFe8qO019069@zion.cs.uiuc.edu> Author: lattner Date: Mon Oct 26 10:40:07 2009 New Revision: 85119 URL: http://llvm.org/viewvc/llvm-project?rev=85119&view=rev Log: reapply r85085 with a bugfix to avoid infinite looping. All of the 'demorgan' related xforms need to use dyn_castNotVal, not m_Not. Modified: llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp llvm/trunk/test/Transforms/InstCombine/or.ll Modified: llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp?rev=85119&r1=85118&r2=85119&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp (original) +++ llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp Mon Oct 26 10:40:07 2009 @@ -631,9 +631,32 @@ return 0; } -static inline Value *dyn_castNotVal(Value *V) { +/// isFreeToInvert - Return true if the specified value is free to invert (apply +/// ~ to). This happens in cases where the ~ can be eliminated. +static inline bool isFreeToInvert(Value *V) { + // ~(~(X)) -> X. if (BinaryOperator::isNot(V)) - return BinaryOperator::getNotArgument(V); + return true; + + // Constants can be considered to be not'ed values. + if (isa(V)) + return true; + + // Compares can be inverted if they have a single use. + if (CmpInst *CI = dyn_cast(V)) + return CI->hasOneUse(); + + return false; +} + +static inline Value *dyn_castNotVal(Value *V) { + // If this is not(not(x)) don't return that this is a not: we want the two + // not's to be folded first. + if (BinaryOperator::isNot(V)) { + Value *Operand = BinaryOperator::getNotArgument(V); + if (!isFreeToInvert(Operand)) + return Operand; + } // Constants can be considered to be not'ed values... if (ConstantInt *C = dyn_cast(V)) @@ -641,6 +664,8 @@ return 0; } + + // dyn_castFoldableMul - If this value is a multiply that can be folded into // other computations (because it has a constant operand), return the // non-constant operand of the multiply, and set CST to point to the multiplier. @@ -4166,7 +4191,7 @@ if (Instruction *CastOp = dyn_cast(CI->getOperand(0))) { if ((isa(CI) || isa(CI)) && CastOp->getNumOperands() == 2) - if (ConstantInt *AndCI = dyn_cast(CastOp->getOperand(1))) { + if (ConstantInt *AndCI =dyn_cast(CastOp->getOperand(1))){ if (CastOp->getOpcode() == Instruction::And) { // Change: and (cast (and X, C1) to T), C2 // into : and (cast X to T), trunc_or_bitcast(C1)&C2 @@ -4960,14 +4985,14 @@ if (Ret) return Ret; } - if (match(Op0, m_Not(m_Value(A)))) { // ~A | Op1 + if ((A = dyn_castNotVal(Op0))) { // ~A | Op1 if (A == Op1) // ~A | A == -1 return ReplaceInstUsesWith(I, Constant::getAllOnesValue(I.getType())); } else { A = 0; } // Note, A is still live here! - if (match(Op1, m_Not(m_Value(B)))) { // Op0 | ~B + if ((B = dyn_castNotVal(Op1))) { // Op0 | ~B if (Op0 == B) return ReplaceInstUsesWith(I, Constant::getAllOnesValue(I.getType())); @@ -5064,12 +5089,13 @@ // Is this a ~ operation? if (Value *NotOp = dyn_castNotVal(&I)) { - // ~(~X & Y) --> (X | ~Y) - De Morgan's Law - // ~(~X | Y) === (X & ~Y) - De Morgan's Law if (BinaryOperator *Op0I = dyn_cast(NotOp)) { if (Op0I->getOpcode() == Instruction::And || Op0I->getOpcode() == Instruction::Or) { - if (dyn_castNotVal(Op0I->getOperand(1))) Op0I->swapOperands(); + // ~(~X & Y) --> (X | ~Y) - De Morgan's Law + // ~(~X | Y) === (X & ~Y) - De Morgan's Law + if (dyn_castNotVal(Op0I->getOperand(1))) + Op0I->swapOperands(); if (Value *Op0NotVal = dyn_castNotVal(Op0I->getOperand(0))) { Value *NotY = Builder->CreateNot(Op0I->getOperand(1), @@ -5078,6 +5104,19 @@ return BinaryOperator::CreateOr(Op0NotVal, NotY); return BinaryOperator::CreateAnd(Op0NotVal, NotY); } + + // ~(X & Y) --> (~X | ~Y) - De Morgan's Law + // ~(X | Y) === (~X & ~Y) - De Morgan's Law + if (isFreeToInvert(Op0I->getOperand(0)) && + isFreeToInvert(Op0I->getOperand(1))) { + Value *NotX = + Builder->CreateNot(Op0I->getOperand(0), "notlhs"); + Value *NotY = + Builder->CreateNot(Op0I->getOperand(1), "notrhs"); + if (Op0I->getOpcode() == Instruction::And) + return BinaryOperator::CreateOr(NotX, NotY); + return BinaryOperator::CreateAnd(NotX, NotY); + } } } } Modified: llvm/trunk/test/Transforms/InstCombine/or.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/InstCombine/or.ll?rev=85119&r1=85118&r2=85119&view=diff ============================================================================== --- llvm/trunk/test/Transforms/InstCombine/or.ll (original) +++ llvm/trunk/test/Transforms/InstCombine/or.ll Mon Oct 26 10:40:07 2009 @@ -1,7 +1,6 @@ ; This test makes sure that these instructions are properly eliminated. ; ; RUN: opt < %s -instcombine -S | FileCheck %s -; XFAIL: * define i32 @test1(i32 %A) { %B = or i32 %A, 0 From gohman at apple.com Mon Oct 26 10:51:41 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 08:51:41 -0700 Subject: [llvm-commits] [llvm] r85017 - in /llvm/trunk/test/Transforms: Inline/callgraph-update.ll JumpThreading/no-irreducible-loops.ll LICM/Preserve-LCSSA.ll LoopRotate/2009-01-25-SingleEntryPhi.ll LoopRotate/LRCrash-1.ll LoopRotate/LRCrash-2.ll LoopRotate/LRCrash-3.ll LoopRotate/LRCrash-4.ll LoopRotate/LRCrash-5.ll LoopRotate/PhiRename-1.ll LoopRotate/PhiSelfRefernce-1.ll LoopRotate/pr2639.ll LoopRotate/preserve-scev.ll LoopSimplify/merge-exits.ll In-Reply-To: <4AE41298.4050102@free.fr> References: <200910242323.n9ONN5fF025552@zion.cs.uiuc.edu> <4AE41298.4050102@free.fr> Message-ID: On Oct 25, 2009, at 1:55 AM, Duncan Sands wrote: > Hi Dan, > >> Make these tests more interesting by using >> -verify-dom-info and -verify-loop-info, which enable additional >> (expensive) consistency checks. > > maybe also these checks can be turned on automatically after loop > passes if LLVM is built with expensive checking turned on. Trying > to bootstrap gcc with an expensive checks build can be informative :) These options are already enabled by default in expensive checks builds. Thanks, Dan From gohman at apple.com Mon Oct 26 10:55:25 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 15:55:25 -0000 Subject: [llvm-commits] [llvm] r85120 - /llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp Message-ID: <200910261555.n9QFtPCs020088@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 10:55:24 2009 New Revision: 85120 URL: http://llvm.org/viewvc/llvm-project?rev=85120&view=rev Log: Fix a typo in a comment. Modified: llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp Modified: llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp?rev=85120&r1=85119&r2=85120&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp (original) +++ llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp Mon Oct 26 10:55:24 2009 @@ -211,7 +211,7 @@ for (I = OrigHeader->begin(); PHINode *PN = dyn_cast(I); ++I) PN->removeIncomingValue(PN->getBasicBlockIndex(OrigPreHeader)); - // Now fix up users of the instructions in OrigHeader, insertting PHI nodes + // Now fix up users of the instructions in OrigHeader, inserting PHI nodes // as necessary. SSAUpdater SSA; for (I = OrigHeader->begin(); I != E; ++I) { From gohman at apple.com Mon Oct 26 10:58:33 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 08:58:33 -0700 Subject: [llvm-commits] [llvm] r85016 - /llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp In-Reply-To: <6FE2703A-1167-451D-B151-E84B042B2378@apple.com> References: <200910242319.n9ONJrei025407@zion.cs.uiuc.edu> <6FE2703A-1167-451D-B151-E84B042B2378@apple.com> Message-ID: <7064681D-2A98-432F-A88E-53E851C2FFF4@apple.com> On Oct 24, 2009, at 6:20 PM, Chris Lattner wrote: > > On Oct 24, 2009, at 4:19 PM, Dan Gohman wrote: > >> Author: djg >> Date: Sat Oct 24 18:19:52 2009 >> New Revision: 85016 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=85016&view=rev >> Log: >> Rewrite LoopRotation's SSA updating code using SSAUpdater. > > Awesome, I didn't remember loop rotate had its own ssa update code. > > If, in your travels, you run into any cases where SSAUpdate is a > significant compile time issue, please give me a testcase. I have > many less-than-crazy ideas for speeding it up. One problem I'm seeing is redundant PHIs. Sphereflake for example now gets code like this: %p.013.i = phi %struct.node_t* [ %p.0.be.i, %bb5.backedge.i ], [ %78, %bb9.i ] ; <%struct.node_t*> [#uses=1] %p.012.i = phi %struct.node_t* [ %p.0.be.i, %bb5.backedge.i ], [ %78, %bb9.i ] ; <%struct.node_t*> [#uses=1] %p.011.i = phi %struct.node_t* [ %p.0.be.i, %bb5.backedge.i ], [ %78, %bb9.i ] ; <%struct.node_t*> [#uses=1] %p.010.i = phi %struct.node_t* [ %p.0.be.i, %bb5.backedge.i ], [ %78, %bb9.i ] ; <%struct.node_t*> [#uses=1] %p.09.i = phi %struct.node_t* [ %p.0.be.i, %bb5.backedge.i ], [ %78, %bb9.i ] ; <%struct.node_t*> [#uses=7] Indvars can't eliminate these because they aren't a function of the canonical induction variable. These PHIs stick around and end up being allocated registers, which is suboptimal. > >> + >> + // Now fix up users of the instructions in OrigHeader, >> insertting PHI nodes > > typo insertting. Fixed. > >> + // as necessary. >> + SSAUpdater SSA; >> + for (I = OrigHeader->begin(); I != E; ++I) { >> + Value *OrigHeaderVal = I; > > Most instructions only have uses within their block and these don't > need to be rewritten. It would improve efficiency to do something > like this if it is safe: > if (I->hasOneUse() && I->use_back()->getParent() == I && > !isa(I->use_back())) > continue; The code already avoids calling RewriteUse in these cases. Are you concerned about the SSAUpdater::Initialize and AddAvailableValue calls? Dan From daniel at zuster.org Mon Oct 26 11:02:25 2009 From: daniel at zuster.org (Daniel Dunbar) Date: Mon, 26 Oct 2009 09:02:25 -0700 Subject: [llvm-commits] [llvm] r83424 - /llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp In-Reply-To: <200910062226.n96MQgaB016790@zion.cs.uiuc.edu> References: <200910062226.n96MQgaB016790@zion.cs.uiuc.edu> Message-ID: <6a8523d60910260902i74d4d85dje53cb78d6f7fa5b@mail.gmail.com> Hi Kevin, A few comments inline: On Tue, Oct 6, 2009 at 3:26 PM, Kevin Enderby wrote: > Author: enderby > Date: Tue Oct ?6 17:26:42 2009 > New Revision: 83424 > > URL: http://llvm.org/viewvc/llvm-project?rev=83424&view=rev > Log: > Added bits of the ARM target assembler to llvm-mc to parse some load instruction > operands. ?Some parsing of arm memory operands for preindexing and postindexing > forms including with register controled shifts. ?This is a work in progress. > > Modified: > ? ?llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp > > Modified: llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp?rev=83424&r1=83423&r2=83424&view=diff > > ============================================================================== > --- llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp (original) > +++ llvm/trunk/lib/Target/ARM/AsmParser/ARMAsmParser.cpp Tue Oct ?6 17:26:42 2009 > @@ -23,6 +23,15 @@ > ?namespace { > ?struct ARMOperand; > > +// The shift types for register controlled shifts in arm memory addressing > +enum ShiftType { > + ?Lsl, > + ?Lsr, > + ?Asr, > + ?Ror, > + ?Rrx > +}; > + > ?class ARMAsmParser : public TargetAsmParser { > ? MCAsmParser &Parser; > > @@ -35,8 +44,31 @@ > > ? bool Error(SMLoc L, const Twine &Msg) { return Parser.Error(L, Msg); } > > + ?bool ParseRegister(ARMOperand &Op); > + > + ?bool ParseMemory(ARMOperand &Op); > + > + ?bool ParseShift(enum ShiftType *St, const MCExpr *ShiftAmount); > + > + ?bool ParseOperand(ARMOperand &Op); > + > ? bool ParseDirectiveWord(unsigned Size, SMLoc L); > > + ?// TODO - For now hacked versions of the next two are in here in this file to > + ?// allow some parser testing until the table gen versions are implemented. > + > + ?/// @name Auto-generated Match Functions > + ?/// { > + ?bool MatchInstruction(SmallVectorImpl &Operands, > + ? ? ? ? ? ? ? ? ? ? ? ?MCInst &Inst); > + > + ?/// MatchRegisterName - Match the given string to a register name, or 0 if > + ?/// there is no match. > + ?unsigned MatchRegisterName(const StringRef &Name); > + > + ?/// } > + > + > ?public: > ? ARMAsmParser(const Target &T, MCAsmParser &_Parser) > ? ? : TargetAsmParser(T), Parser(_Parser) {} > @@ -48,9 +80,380 @@ > > ?} // end anonymous namespace > > +namespace { > + > +/// ARMOperand - Instances of this class represent a parsed ARM machine > +/// instruction. > +struct ARMOperand { > + ?enum { > + ? ?Token, > + ? ?Register, > + ? ?Memory > + ?} Kind; > + > + > + ?union { > + ? ?struct { > + ? ? ?const char *Data; > + ? ? ?unsigned Length; > + ? ?} Tok; > + > + ? ?struct { > + ? ? ?unsigned RegNum; > + ? ?} Reg; > + > + ? ?// This is for all forms of ARM address expressions > + ? ?struct { > + ? ? ?unsigned BaseRegNum; > + ? ? ?bool OffsetIsReg; > + ? ? ?const MCExpr *Offset; // used when OffsetIsReg is false > + ? ? ?unsigned OffsetRegNum; // used when OffsetIsReg is true > + ? ? ?bool OffsetRegShifted; // only used when OffsetIsReg is true > + ? ? ?enum ShiftType ShiftType; ?// used when OffsetRegShifted is true > + ? ? ?const MCExpr *ShiftAmount; // used when OffsetRegShifted is true > + ? ? ?bool Preindexed; > + ? ? ?bool Postindexed; > + ? ? ?bool Negative; // only used when OffsetIsReg is true > + ? ? ?bool Writeback; > + ? ?} Mem; It probably makes sense to reorder these fields to make the struct smaller (bool -> 'unsigned foo : 1', and make MCExpr* fields adjacent). > +// Try to parse a register name. ?The token must be an Identifier when called, > +// and if it is a register name a Reg operand is created, the token is eaten > +// and false is returned. ?Else true is returned and no token is eaten. > +// TODO this is likely to change to allow different register types and or to > +// parse for a specific register type. Please use '///' instead of '//' so the comment gets picked up by doxygen. This applies in a number of places in this file. > +bool ARMAsmParser::ParseRegister(ARMOperand &Op) { > + ?const AsmToken &Tok = getLexer().getTok(); > + ?assert(Tok.is(AsmToken::Identifier) && "Token is not an Identifier"); > + > + ?// FIXME: Validate register for the current architecture; we have to do > + ?// validation later, so maybe there is no need for this here. > + ?unsigned RegNum; > + > + ?RegNum = MatchRegisterName(Tok.getString()); > + ?if (RegNum == 0) > + ? ?return true; > + > + ?Op = ARMOperand::CreateReg(RegNum); > + ?getLexer().Lex(); // Eat identifier token. > + > + ?return false; > +} Since this is a "try parse" routine, and doesn't emit errors or consume tokens on failure, it should be called MaybeParseRegister. > + > +// Try to parse an arm memory expression. ?It must start with a '[' token. > +// TODO Only preindexing and postindexing addressing are started, unindexed > +// with option, etc are still to do. > +bool ARMAsmParser::ParseMemory(ARMOperand &Op) { > + ?const AsmToken &LBracTok = getLexer().getTok(); > + ?assert(LBracTok.is(AsmToken::LBrac) && "Token is not an Left Bracket"); > + ?getLexer().Lex(); // Eat left bracket token. > + > + ?const AsmToken &BaseRegTok = getLexer().getTok(); > + ?if (BaseRegTok.isNot(AsmToken::Identifier)) > + ? ?return Error(BaseRegTok.getLoc(), "register expected"); > + ?unsigned BaseRegNum = MatchRegisterName(BaseRegTok.getString()); > + ?if (BaseRegNum == 0) > + ? ?return Error(BaseRegTok.getLoc(), "register expected"); Shouldn't this just call ParseRegister? > + ?getLexer().Lex(); // Eat identifier token. > + > + ?bool Preindexed = false; > + ?bool Postindexed = false; > + ?bool OffsetIsReg = false; > + ?bool Negative = false; > + ?bool Writeback = false; > + > + ?// First look for preindexed address forms: > + ?// ?[Rn, +/-Rm] > + ?// ?[Rn, #offset] > + ?// ?[Rn, +/-Rm, shift] > + ?// that is after the "[Rn" we now have see if the next token is a comma. > + ?const AsmToken &Tok = getLexer().getTok(); > + ?if (Tok.is(AsmToken::Comma)) { > + ? ?Preindexed = true; > + ? ?getLexer().Lex(); // Eat comma token. > + > + ? ?const AsmToken &NextTok = getLexer().getTok(); > + ? ?if (NextTok.is(AsmToken::Plus)) > + ? ? ?getLexer().Lex(); // Eat plus token. > + ? ?else if (NextTok.is(AsmToken::Minus)) { > + ? ? ?Negative = true; > + ? ? ?getLexer().Lex(); // Eat minus token > + ? ?} > + > + ? ?// See if there is a register following the "[Rn," we have so far. > + ? ?const AsmToken &OffsetRegTok = getLexer().getTok(); > + ? ?unsigned OffsetRegNum = MatchRegisterName(OffsetRegTok.getString()); Again, I think this should just call ParseRegister. Is a plus/minus not required here? This code will accept [Rn, Rm], is that intended? If so the comment above listing the forms should be updated. > + ? ?bool OffsetRegShifted = false; > + ? ?enum ShiftType ShiftType; > + ? ?const MCExpr *ShiftAmount; > + ? ?const MCExpr *Offset; > + ? ?if (OffsetRegNum != 0) { > + ? ? ?OffsetIsReg = true; > + ? ? ?getLexer().Lex(); // Eat identifier token for the offset register. > + ? ? ?// Look for a comma then a shift > + ? ? ?const AsmToken &Tok = getLexer().getTok(); > + ? ? ?if (Tok.is(AsmToken::Comma)) { > + ? ? ? ?getLexer().Lex(); // Eat comma token. > + > + ? ? ? ?const AsmToken &Tok = getLexer().getTok(); > + ? ? ? ?if (ParseShift(&ShiftType, ShiftAmount)) > + ? ? ? ? ?return Error(Tok.getLoc(), "shift expected"); > + ? ? ? ?OffsetRegShifted = true; > + ? ? ?} > + ? ?} > + ? ?else { // "[Rn," we have so far was not followed by "Rm" > + ? ? ?// Look for #offset following the "[Rn," > + ? ? ?const AsmToken &HashTok = getLexer().getTok(); > + ? ? ?if (HashTok.isNot(AsmToken::Hash)) > + ? ? ? ?return Error(HashTok.getLoc(), "'#' expected"); > + ? ? ?getLexer().Lex(); // Eat hash token. > + > + ? ? ?if (getParser().ParseExpression(Offset)) > + ? ? ? return true; > + ? ?} > + ? ?const AsmToken &RBracTok = getLexer().getTok(); > + ? ?if (RBracTok.isNot(AsmToken::RBrac)) > + ? ? ?return Error(RBracTok.getLoc(), "']' expected"); > + ? ?getLexer().Lex(); // Eat right bracket token. > + > + ? ?const AsmToken &ExclaimTok = getLexer().getTok(); > + ? ?if (ExclaimTok.is(AsmToken::Exclaim)) { > + ? ? ?Writeback = true; > + ? ? ?getLexer().Lex(); // Eat exclaim token > + ? ?} Can you please include an example of this form in the list of forms this routine is parsing? > + ? ?Op = ARMOperand::CreateMem(BaseRegNum, OffsetIsReg, Offset, OffsetRegNum, > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? OffsetRegShifted, ShiftType, ShiftAmount, > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Preindexed, Postindexed, Negative, Writeback); > + ? ?return false; > + ?} > + ?// The "[Rn" we have so far was not followed by a comma. > + ?else if (Tok.is(AsmToken::RBrac)) { > + ? ?// This is a post indexing addressing forms: > + ? ?// ?[Rn], #offset > + ? ?// ?[Rn], +/-Rm > + ? ?// ?[Rn], +/-Rm, shift > + ? ?// that is a ']' follows after the "[Rn". > + ? ?Postindexed = true; > + ? ?Writeback = true; > + ? ?getLexer().Lex(); // Eat right bracket token. > + > + ? ?const AsmToken &CommaTok = getLexer().getTok(); > + ? ?if (CommaTok.isNot(AsmToken::Comma)) > + ? ? ?return Error(CommaTok.getLoc(), "',' expected"); > + ? ?getLexer().Lex(); // Eat comma token. > + > + ? ?const AsmToken &NextTok = getLexer().getTok(); > + ? ?if (NextTok.is(AsmToken::Plus)) > + ? ? ?getLexer().Lex(); // Eat plus token. > + ? ?else if (NextTok.is(AsmToken::Minus)) { > + ? ? ?Negative = true; > + ? ? ?getLexer().Lex(); // Eat minus token > + ? ?} > + > + ? ?// See if there is a register following the "[Rn]," we have so far. > + ? ?const AsmToken &OffsetRegTok = getLexer().getTok(); > + ? ?unsigned OffsetRegNum = MatchRegisterName(OffsetRegTok.getString()); > + ? ?bool OffsetRegShifted = false; > + ? ?enum ShiftType ShiftType; > + ? ?const MCExpr *ShiftAmount; > + ? ?const MCExpr *Offset; > + ? ?if (OffsetRegNum != 0) { > + ? ? ?OffsetIsReg = true; > + ? ? ?getLexer().Lex(); // Eat identifier token for the offset register. > + ? ? ?// Look for a comma then a shift > + ? ? ?const AsmToken &Tok = getLexer().getTok(); > + ? ? ?if (Tok.is(AsmToken::Comma)) { > + ? ? ? ?getLexer().Lex(); // Eat comma token. > + > + ? ? ? ?const AsmToken &Tok = getLexer().getTok(); > + ? ? ? ?if (ParseShift(&ShiftType, ShiftAmount)) > + ? ? ? ? ?return Error(Tok.getLoc(), "shift expected"); > + ? ? ? ?OffsetRegShifted = true; > + ? ? ?} > + ? ?} > + ? ?else { // "[Rn]," we have so far was not followed by "Rm" > + ? ? ?// Look for #offset following the "[Rn]," > + ? ? ?const AsmToken &HashTok = getLexer().getTok(); > + ? ? ?if (HashTok.isNot(AsmToken::Hash)) > + ? ? ? ?return Error(HashTok.getLoc(), "'#' expected"); > + ? ? ?getLexer().Lex(); // Eat hash token. > + > + ? ? ?if (getParser().ParseExpression(Offset)) > + ? ? ? return true; > + ? ?} > + ? ?Op = ARMOperand::CreateMem(BaseRegNum, OffsetIsReg, Offset, OffsetRegNum, > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? OffsetRegShifted, ShiftType, ShiftAmount, > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Preindexed, Postindexed, Negative, Writeback); > + ? ?return false; > + ?} > + > + ?return true; > +} I think it might be cleaner to factor this routine into a part that parses: ',' #offset ',' +/-Rm ',' +/-Rm, shift and then reuse that for both the pre and post indexed forms. > +/// ParseShift as one of these two: > +/// ? ( lsl | lsr | asr | ror ) , # shift_amount > +/// ? rrx > +/// and returns true if it parses a shift otherwise it returns false. > +bool ARMAsmParser::ParseShift(ShiftType *St, const MCExpr *ShiftAmount) { This should take a reference to ShiftType instead of taking a pointer to it (convention). > + ?const AsmToken &Tok = getLexer().getTok(); > + ?if (Tok.isNot(AsmToken::Identifier)) > + ? ?return true; > + ?const StringRef &ShiftName = Tok.getString(); > + ?if (ShiftName == "lsl" || ShiftName == "LSL") > + ? ?*St = Lsl; > + ?else if (ShiftName == "lsr" || ShiftName == "LSR") > + ? ?*St = Lsr; > + ?else if (ShiftName == "asr" || ShiftName == "ASR") > + ? ?*St = Asr; > + ?else if (ShiftName == "ror" || ShiftName == "ROR") > + ? ?*St = Ror; > + ?else if (ShiftName == "rrx" || ShiftName == "RRX") > + ? ?*St = Rrx; > + ?else > + ? ?return true; > + ?getLexer().Lex(); // Eat shift type token. > + > + ?// For all but a Rotate right there must be a '#' and a shift amount > + ?if (*St != Rrx) { Chris will be happier if you write this as: -- // Rrx stands alone. if (*St == Rrx) return false; // Otherwise, there must be a '#' and a shift amount. ... -- to reduce the nesting. :) > + ? ?// Look for # following the shift type > + ? ?const AsmToken &HashTok = getLexer().getTok(); > + ? ?if (HashTok.isNot(AsmToken::Hash)) > + ? ? ?return Error(HashTok.getLoc(), "'#' expected"); > + ? ?getLexer().Lex(); // Eat hash token. > + > + ? ?if (getParser().ParseExpression(ShiftAmount)) > + ? ? ?return true; > + ?} > + > + ?return false; > +} > + > +// A hack to allow some testing > +unsigned ARMAsmParser::MatchRegisterName(const StringRef &Name) { > + ?if (Name == "r1") > + ? ?return 1; > + ?else if (Name == "r2") > + ? ?return 2; > + ?else if (Name == "r3") > + ? ?return 3; > + ?return 0; > +} > + > +// A hack to allow some testing > +bool ARMAsmParser::MatchInstruction(SmallVectorImpl &Operands, > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?MCInst &Inst) { > + ?struct ARMOperand Op0 = Operands[0]; > + ?assert(Op0.Kind == ARMOperand::Token && "First operand not a Token"); > + ?const StringRef &Mnemonic = Op0.getToken(); > + ?if (Mnemonic == "add" || > + ? ? ?Mnemonic == "ldr") > + ? ?return false; > + > + ?return true; > +} > + > +// TODO - this is a work in progress > +bool ARMAsmParser::ParseOperand(ARMOperand &Op) { > + ?switch (getLexer().getKind()) { > + ?case AsmToken::Identifier: > + ? ?if (!ParseRegister(Op)) > + ? ? ?return false; > + ? ?// TODO parse other operands that start with an identifier > + ? ?return true; > + ?case AsmToken::LBrac: > + ? ?if (!ParseMemory(Op)) > + ? ? ?return false; > + ?default: > + ? ?return true; > + ?} > +} > + > ?bool ARMAsmParser::ParseInstruction(const StringRef &Name, MCInst &Inst) { > + ?SmallVector Operands; > + > + ?Operands.push_back(ARMOperand::CreateToken(Name)); > + > ? SMLoc Loc = getLexer().getTok().getLoc(); > - ?Error(Loc, "ARMAsmParser::ParseInstruction currently unimplemented"); > + ?if (getLexer().isNot(AsmToken::EndOfStatement)) { > + > + ? ?// Read the first operand. > + ? ?Operands.push_back(ARMOperand()); > + ? ?if (ParseOperand(Operands.back())) > + ? ? ?return true; > + > + ? ?while (getLexer().is(AsmToken::Comma)) { > + ? ? ?getLexer().Lex(); ?// Eat the comma. > + > + ? ? ?// Parse and remember the operand. > + ? ? ?Operands.push_back(ARMOperand()); > + ? ? ?if (ParseOperand(Operands.back())) > + ? ? ? ?return true; > + ? ?} > + ?} > + ?if (!MatchInstruction(Operands, Inst)) > + ? ?return false; > + > + ?Error(Loc, "ARMAsmParser::ParseInstruction only partly implemented"); > ? return true; > ?} Thanks! - Daniel From dpatel at apple.com Mon Oct 26 11:54:36 2009 From: dpatel at apple.com (Devang Patel) Date: Mon, 26 Oct 2009 16:54:36 -0000 Subject: [llvm-commits] [llvm] r85126 - in /llvm/trunk: include/llvm/Analysis/DebugInfo.h lib/Analysis/DebugInfo.cpp Message-ID: <200910261654.n9QGsapO023661@zion.cs.uiuc.edu> Author: dpatel Date: Mon Oct 26 11:54:35 2009 New Revision: 85126 URL: http://llvm.org/viewvc/llvm-project?rev=85126&view=rev Log: Add support to encode type info using llvm::Constant. Patch by Talin! Modified: llvm/trunk/include/llvm/Analysis/DebugInfo.h llvm/trunk/lib/Analysis/DebugInfo.cpp Modified: llvm/trunk/include/llvm/Analysis/DebugInfo.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Analysis/DebugInfo.h?rev=85126&r1=85125&r2=85126&view=diff ============================================================================== --- llvm/trunk/include/llvm/Analysis/DebugInfo.h (original) +++ llvm/trunk/include/llvm/Analysis/DebugInfo.h Mon Oct 26 11:54:35 2009 @@ -514,6 +514,13 @@ uint64_t OffsetInBits, unsigned Flags, unsigned Encoding); + /// CreateBasicType - Create a basic type like int, float, etc. + DIBasicType CreateBasicTypeEx(DIDescriptor Context, StringRef Name, + DICompileUnit CompileUnit, unsigned LineNumber, + Constant *SizeInBits, Constant *AlignInBits, + Constant *OffsetInBits, unsigned Flags, + unsigned Encoding); + /// CreateDerivedType - Create a derived type like const qualified type, /// pointer, typedef, etc. DIDerivedType CreateDerivedType(unsigned Tag, DIDescriptor Context, @@ -524,6 +531,16 @@ uint64_t OffsetInBits, unsigned Flags, DIType DerivedFrom); + /// CreateDerivedType - Create a derived type like const qualified type, + /// pointer, typedef, etc. + DIDerivedType CreateDerivedTypeEx(unsigned Tag, DIDescriptor Context, + StringRef Name, + DICompileUnit CompileUnit, + unsigned LineNumber, + Constant *SizeInBits, Constant *AlignInBits, + Constant *OffsetInBits, unsigned Flags, + DIType DerivedFrom); + /// CreateCompositeType - Create a composite type like array, struct, etc. DICompositeType CreateCompositeType(unsigned Tag, DIDescriptor Context, StringRef Name, @@ -536,6 +553,18 @@ DIArray Elements, unsigned RunTimeLang = 0); + /// CreateCompositeType - Create a composite type like array, struct, etc. + DICompositeType CreateCompositeTypeEx(unsigned Tag, DIDescriptor Context, + StringRef Name, + DICompileUnit CompileUnit, + unsigned LineNumber, + Constant *SizeInBits, + Constant *AlignInBits, + Constant *OffsetInBits, unsigned Flags, + DIType DerivedFrom, + DIArray Elements, + unsigned RunTimeLang = 0); + /// CreateSubprogram - Create a new descriptor for the specified subprogram. /// See comments in DISubprogram for descriptions of these fields. DISubprogram CreateSubprogram(DIDescriptor Context, StringRef Name, Modified: llvm/trunk/lib/Analysis/DebugInfo.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/DebugInfo.cpp?rev=85126&r1=85125&r2=85126&view=diff ============================================================================== --- llvm/trunk/lib/Analysis/DebugInfo.cpp (original) +++ llvm/trunk/lib/Analysis/DebugInfo.cpp Mon Oct 26 11:54:35 2009 @@ -695,6 +695,32 @@ return DIBasicType(MDNode::get(VMContext, &Elts[0], 10)); } + +/// CreateBasicType - Create a basic type like int, float, etc. +DIBasicType DIFactory::CreateBasicTypeEx(DIDescriptor Context, + StringRef Name, + DICompileUnit CompileUnit, + unsigned LineNumber, + Constant *SizeInBits, + Constant *AlignInBits, + Constant *OffsetInBits, unsigned Flags, + unsigned Encoding) { + Value *Elts[] = { + GetTagConstant(dwarf::DW_TAG_base_type), + Context.getNode(), + MDString::get(VMContext, Name), + CompileUnit.getNode(), + ConstantInt::get(Type::getInt32Ty(VMContext), LineNumber), + SizeInBits, + AlignInBits, + OffsetInBits, + ConstantInt::get(Type::getInt32Ty(VMContext), Flags), + ConstantInt::get(Type::getInt32Ty(VMContext), Encoding) + }; + return DIBasicType(MDNode::get(VMContext, &Elts[0], 10)); +} + + /// CreateDerivedType - Create a derived type like const qualified type, /// pointer, typedef, etc. DIDerivedType DIFactory::CreateDerivedType(unsigned Tag, @@ -722,6 +748,35 @@ return DIDerivedType(MDNode::get(VMContext, &Elts[0], 10)); } + +/// CreateDerivedType - Create a derived type like const qualified type, +/// pointer, typedef, etc. +DIDerivedType DIFactory::CreateDerivedTypeEx(unsigned Tag, + DIDescriptor Context, + StringRef Name, + DICompileUnit CompileUnit, + unsigned LineNumber, + Constant *SizeInBits, + Constant *AlignInBits, + Constant *OffsetInBits, + unsigned Flags, + DIType DerivedFrom) { + Value *Elts[] = { + GetTagConstant(Tag), + Context.getNode(), + MDString::get(VMContext, Name), + CompileUnit.getNode(), + ConstantInt::get(Type::getInt32Ty(VMContext), LineNumber), + SizeInBits, + AlignInBits, + OffsetInBits, + ConstantInt::get(Type::getInt32Ty(VMContext), Flags), + DerivedFrom.getNode(), + }; + return DIDerivedType(MDNode::get(VMContext, &Elts[0], 10)); +} + + /// CreateCompositeType - Create a composite type like array, struct, etc. DICompositeType DIFactory::CreateCompositeType(unsigned Tag, DIDescriptor Context, @@ -754,6 +809,38 @@ } +/// CreateCompositeType - Create a composite type like array, struct, etc. +DICompositeType DIFactory::CreateCompositeTypeEx(unsigned Tag, + DIDescriptor Context, + StringRef Name, + DICompileUnit CompileUnit, + unsigned LineNumber, + Constant *SizeInBits, + Constant *AlignInBits, + Constant *OffsetInBits, + unsigned Flags, + DIType DerivedFrom, + DIArray Elements, + unsigned RuntimeLang) { + + Value *Elts[] = { + GetTagConstant(Tag), + Context.getNode(), + MDString::get(VMContext, Name), + CompileUnit.getNode(), + ConstantInt::get(Type::getInt32Ty(VMContext), LineNumber), + SizeInBits, + AlignInBits, + OffsetInBits, + ConstantInt::get(Type::getInt32Ty(VMContext), Flags), + DerivedFrom.getNode(), + Elements.getNode(), + ConstantInt::get(Type::getInt32Ty(VMContext), RuntimeLang) + }; + return DICompositeType(MDNode::get(VMContext, &Elts[0], 12)); +} + + /// CreateSubprogram - Create a new descriptor for the specified subprogram. /// See comments in DISubprogram for descriptions of these fields. This /// method does not unique the generated descriptors. From david_goodwin at apple.com Mon Oct 26 11:59:04 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 16:59:04 -0000 Subject: [llvm-commits] [llvm] r85127 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/CMakeLists.txt lib/CodeGen/CriticalAntiDepBreaker.cpp lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp lib/Target/ARM/ARMSubtarget.h test/CodeGen/X86/break-anti-dependencies.ll Message-ID: <200910261659.n9QGx4Yj023936@zion.cs.uiuc.edu> Author: david_goodwin Date: Mon Oct 26 11:59:04 2009 New Revision: 85127 URL: http://llvm.org/viewvc/llvm-project?rev=85127&view=rev Log: Break anti-dependence breaking out into its own class. Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp llvm/trunk/lib/Target/ARM/ARMSubtarget.h llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85127&view=auto ============================================================================== --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (added) +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 11:59:04 2009 @@ -0,0 +1,56 @@ +//=- llvm/CodeGen/AntiDepBreaker.h - Anti-Dependence Breaking -*- C++ -*-=// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements the AntiDepBreaker class, which implements +// anti-dependence breaking heuristics for post-register-allocation scheduling. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_CODEGEN_ANTIDEPBREAKER_H +#define LLVM_CODEGEN_ANTIDEPBREAKER_H + +#include "llvm/CodeGen/MachineBasicBlock.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/CodeGen/MachineFunction.h" +#include "llvm/CodeGen/MachineRegisterInfo.h" +#include "llvm/CodeGen/ScheduleDAG.h" +#include "llvm/Target/TargetRegisterInfo.h" + +namespace llvm { + +/// AntiDepBreaker - This class works into conjunction with the +/// post-RA scheduler to rename registers to break register +/// anti-dependencies. +class AntiDepBreaker { +public: + /// Start - Initialize anti-dep breaking for a new basic block. + virtual void StartBlock(MachineBasicBlock *BB) =0; + + /// BreakAntiDependencies - Identifiy anti-dependencies within a + /// basic-block region and break them by renaming registers. Return + /// the number of anti-dependencies broken. + /// + virtual unsigned BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex) =0; + + /// Observe - Update liveness information to account for the current + /// instruction, which will not be scheduled. + /// + virtual void Observe(MachineInstr *MI, unsigned Count, + unsigned InsertPosIndex) =0; + + /// Finish - Finish anti-dep breaking for a basic block. + virtual void FinishBlock() =0; +}; + +} + +#endif Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85127&r1=85126&r2=85127&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 11:59:04 2009 @@ -1,6 +1,7 @@ add_llvm_library(LLVMCodeGen BranchFolding.cpp CodePlacementOpt.cpp + CriticalAntiDepBreaker.cpp DeadMachineInstructionElim.cpp DwarfEHPrepare.cpp ELFCodeEmitter.cpp Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp?rev=85127&view=auto ============================================================================== --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp (added) +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp Mon Oct 26 11:59:04 2009 @@ -0,0 +1,539 @@ +//===----- CriticalAntiDepBreaker.cpp - Anti-dep breaker -------- ---------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements the CriticalAntiDepBreaker class, which +// implements register anti-dependence breaking along a blocks +// critical path during post-RA scheduler. +// +//===----------------------------------------------------------------------===// + +#define DEBUG_TYPE "critical-antidep" +#include "CriticalAntiDepBreaker.h" +#include "llvm/CodeGen/MachineBasicBlock.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/Target/TargetMachine.h" +#include "llvm/Target/TargetRegisterInfo.h" +#include "llvm/Support/Debug.h" +#include "llvm/Support/ErrorHandling.h" +#include "llvm/Support/raw_ostream.h" + +using namespace llvm; + +CriticalAntiDepBreaker:: +CriticalAntiDepBreaker(MachineFunction& MFi) : + AntiDepBreaker(), MF(MFi), + MRI(MF.getRegInfo()), + TRI(MF.getTarget().getRegisterInfo()), + AllocatableSet(TRI->getAllocatableSet(MF)) +{ +} + +CriticalAntiDepBreaker::~CriticalAntiDepBreaker() { +} + +void CriticalAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { + // Clear out the register class data. + std::fill(Classes, array_endof(Classes), + static_cast(0)); + + // Initialize the indices to indicate that no registers are live. + std::fill(KillIndices, array_endof(KillIndices), ~0u); + std::fill(DefIndices, array_endof(DefIndices), BB->size()); + + // Clear "do not change" set. + KeepRegs.clear(); + + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); + + // Determine the live-out physregs for this block. + if (IsReturnBlock) { + // In a return block, examine the function live-out regs. + for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), + E = MRI.liveout_end(); I != E; ++I) { + unsigned Reg = *I; + Classes[Reg] = reinterpret_cast(-1); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + Classes[AliasReg] = reinterpret_cast(-1); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } + } else { + // In a non-return block, examine the live-in regs of all successors. + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), + SE = BB->succ_end(); SI != SE; ++SI) + for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), + E = (*SI)->livein_end(); I != E; ++I) { + unsigned Reg = *I; + Classes[Reg] = reinterpret_cast(-1); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + Classes[AliasReg] = reinterpret_cast(-1); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } + } + + // Mark live-out callee-saved registers. In a return block this is + // all callee-saved registers. In non-return this is any + // callee-saved register that is not saved in the prolog. + const MachineFrameInfo *MFI = MF.getFrameInfo(); + BitVector Pristine = MFI->getPristineRegs(BB); + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { + unsigned Reg = *I; + if (!IsReturnBlock && !Pristine.test(Reg)) continue; + Classes[Reg] = reinterpret_cast(-1); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + Classes[AliasReg] = reinterpret_cast(-1); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } +} + +void CriticalAntiDepBreaker::FinishBlock() { + RegRefs.clear(); + KeepRegs.clear(); +} + +void CriticalAntiDepBreaker::Observe(MachineInstr *MI, unsigned Count, + unsigned InsertPosIndex) { + assert(Count < InsertPosIndex && "Instruction index out of expected range!"); + + // Any register which was defined within the previous scheduling region + // may have been rescheduled and its lifetime may overlap with registers + // in ways not reflected in our current liveness state. For each such + // register, adjust the liveness state to be conservatively correct. + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) + if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { + assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); + // Mark this register to be non-renamable. + Classes[Reg] = reinterpret_cast(-1); + // Move the def index to the end of the previous region, to reflect + // that the def could theoretically have been scheduled at the end. + DefIndices[Reg] = InsertPosIndex; + } + + PrescanInstruction(MI); + ScanInstruction(MI, Count); +} + +/// CriticalPathStep - Return the next SUnit after SU on the bottom-up +/// critical path. +static SDep *CriticalPathStep(SUnit *SU) { + SDep *Next = 0; + unsigned NextDepth = 0; + // Find the predecessor edge with the greatest depth. + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); + P != PE; ++P) { + SUnit *PredSU = P->getSUnit(); + unsigned PredLatency = P->getLatency(); + unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; + // In the case of a latency tie, prefer an anti-dependency edge over + // other types of edges. + if (NextDepth < PredTotalLatency || + (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { + NextDepth = PredTotalLatency; + Next = &*P; + } + } + return Next; +} + +void CriticalAntiDepBreaker::PrescanInstruction(MachineInstr *MI) { + // Scan the register operands for this instruction and update + // Classes and RegRefs. + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + const TargetRegisterClass *NewRC = 0; + + if (i < MI->getDesc().getNumOperands()) + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); + + // For now, only allow the register to be changed if its register + // class is consistent across all uses. + if (!Classes[Reg] && NewRC) + Classes[Reg] = NewRC; + else if (!NewRC || Classes[Reg] != NewRC) + Classes[Reg] = reinterpret_cast(-1); + + // Now check for aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + // If an alias of the reg is used during the live range, give up. + // Note that this allows us to skip checking if AntiDepReg + // overlaps with any of the aliases, among other things. + unsigned AliasReg = *Alias; + if (Classes[AliasReg]) { + Classes[AliasReg] = reinterpret_cast(-1); + Classes[Reg] = reinterpret_cast(-1); + } + } + + // If we're still willing to consider this register, note the reference. + if (Classes[Reg] != reinterpret_cast(-1)) + RegRefs.insert(std::make_pair(Reg, &MO)); + + // It's not safe to change register allocation for source operands of + // that have special allocation requirements. + if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { + if (KeepRegs.insert(Reg)) { + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) + KeepRegs.insert(*Subreg); + } + } + } +} + +void CriticalAntiDepBreaker::ScanInstruction(MachineInstr *MI, + unsigned Count) { + // Update liveness. + // Proceding upwards, registers that are defed but not used in this + // instruction are now dead. + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + if (!MO.isDef()) continue; + // Ignore two-addr defs. + if (MI->isRegTiedToUseOperand(i)) continue; + + DefIndices[Reg] = Count; + KillIndices[Reg] = ~0u; + assert(((KillIndices[Reg] == ~0u) != + (DefIndices[Reg] == ~0u)) && + "Kill and Def maps aren't consistent for Reg!"); + KeepRegs.erase(Reg); + Classes[Reg] = 0; + RegRefs.erase(Reg); + // Repeat, for all subregs. + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) { + unsigned SubregReg = *Subreg; + DefIndices[SubregReg] = Count; + KillIndices[SubregReg] = ~0u; + KeepRegs.erase(SubregReg); + Classes[SubregReg] = 0; + RegRefs.erase(SubregReg); + } + // Conservatively mark super-registers as unusable. + for (const unsigned *Super = TRI->getSuperRegisters(Reg); + *Super; ++Super) { + unsigned SuperReg = *Super; + Classes[SuperReg] = reinterpret_cast(-1); + } + } + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + if (!MO.isUse()) continue; + + const TargetRegisterClass *NewRC = 0; + if (i < MI->getDesc().getNumOperands()) + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); + + // For now, only allow the register to be changed if its register + // class is consistent across all uses. + if (!Classes[Reg] && NewRC) + Classes[Reg] = NewRC; + else if (!NewRC || Classes[Reg] != NewRC) + Classes[Reg] = reinterpret_cast(-1); + + RegRefs.insert(std::make_pair(Reg, &MO)); + + // It wasn't previously live but now it is, this is a kill. + if (KillIndices[Reg] == ~0u) { + KillIndices[Reg] = Count; + DefIndices[Reg] = ~0u; + assert(((KillIndices[Reg] == ~0u) != + (DefIndices[Reg] == ~0u)) && + "Kill and Def maps aren't consistent for Reg!"); + } + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + if (KillIndices[AliasReg] == ~0u) { + KillIndices[AliasReg] = Count; + DefIndices[AliasReg] = ~0u; + } + } + } +} + +unsigned +CriticalAntiDepBreaker::findSuitableFreeRegister(unsigned AntiDepReg, + unsigned LastNewReg, + const TargetRegisterClass *RC) { + for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), + RE = RC->allocation_order_end(MF); R != RE; ++R) { + unsigned NewReg = *R; + // Don't replace a register with itself. + if (NewReg == AntiDepReg) continue; + // Don't replace a register with one that was recently used to repair + // an anti-dependence with this AntiDepReg, because that would + // re-introduce that anti-dependence. + if (NewReg == LastNewReg) continue; + // If NewReg is dead and NewReg's most recent def is not before + // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. + assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && + "Kill and Def maps aren't consistent for AntiDepReg!"); + assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && + "Kill and Def maps aren't consistent for NewReg!"); + if (KillIndices[NewReg] != ~0u || + Classes[NewReg] == reinterpret_cast(-1) || + KillIndices[AntiDepReg] > DefIndices[NewReg]) + continue; + return NewReg; + } + + // No registers are free and available! + return 0; +} + +unsigned CriticalAntiDepBreaker:: +BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex) { + // The code below assumes that there is at least one instruction, + // so just duck out immediately if the block is empty. + if (SUnits.empty()) return 0; + + // Find the node at the bottom of the critical path. + SUnit *Max = 0; + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { + SUnit *SU = &SUnits[i]; + if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) + Max = SU; + } + +#ifndef NDEBUG + { + DEBUG(errs() << "Critical path has total latency " + << (Max->getDepth() + Max->Latency) << "\n"); + DEBUG(errs() << "Available regs:"); + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { + if (KillIndices[Reg] == ~0u) + DEBUG(errs() << " " << TRI->getName(Reg)); + } + DEBUG(errs() << '\n'); + } +#endif + + // Track progress along the critical path through the SUnit graph as we walk + // the instructions. + SUnit *CriticalPathSU = Max; + MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); + + // Consider this pattern: + // A = ... + // ... = A + // A = ... + // ... = A + // A = ... + // ... = A + // A = ... + // ... = A + // There are three anti-dependencies here, and without special care, + // we'd break all of them using the same register: + // A = ... + // ... = A + // B = ... + // ... = B + // B = ... + // ... = B + // B = ... + // ... = B + // because at each anti-dependence, B is the first register that + // isn't A which is free. This re-introduces anti-dependencies + // at all but one of the original anti-dependencies that we were + // trying to break. To avoid this, keep track of the most recent + // register that each register was replaced with, avoid + // using it to repair an anti-dependence on the same register. + // This lets us produce this: + // A = ... + // ... = A + // B = ... + // ... = B + // C = ... + // ... = C + // B = ... + // ... = B + // This still has an anti-dependence on B, but at least it isn't on the + // original critical path. + // + // TODO: If we tracked more than one register here, we could potentially + // fix that remaining critical edge too. This is a little more involved, + // because unlike the most recent register, less recent registers should + // still be considered, though only if no other registers are available. + unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; + + // Attempt to break anti-dependence edges on the critical path. Walk the + // instructions from the bottom up, tracking information about liveness + // as we go to help determine which registers are available. + unsigned Broken = 0; + unsigned Count = InsertPosIndex - 1; + for (MachineBasicBlock::iterator I = End, E = Begin; + I != E; --Count) { + MachineInstr *MI = --I; + + // Check if this instruction has a dependence on the critical path that + // is an anti-dependence that we may be able to break. If it is, set + // AntiDepReg to the non-zero register associated with the anti-dependence. + // + // We limit our attention to the critical path as a heuristic to avoid + // breaking anti-dependence edges that aren't going to significantly + // impact the overall schedule. There are a limited number of registers + // and we want to save them for the important edges. + // + // TODO: Instructions with multiple defs could have multiple + // anti-dependencies. The current code here only knows how to break one + // edge per instruction. Note that we'd have to be able to break all of + // the anti-dependencies in an instruction in order to be effective. + unsigned AntiDepReg = 0; + if (MI == CriticalPathMI) { + if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { + SUnit *NextSU = Edge->getSUnit(); + + // Only consider anti-dependence edges. + if (Edge->getKind() == SDep::Anti) { + AntiDepReg = Edge->getReg(); + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); + if (!AllocatableSet.test(AntiDepReg)) + // Don't break anti-dependencies on non-allocatable registers. + AntiDepReg = 0; + else if (KeepRegs.count(AntiDepReg)) + // Don't break anti-dependencies if an use down below requires + // this exact register. + AntiDepReg = 0; + else { + // If the SUnit has other dependencies on the SUnit that it + // anti-depends on, don't bother breaking the anti-dependency + // since those edges would prevent such units from being + // scheduled past each other regardless. + // + // Also, if there are dependencies on other SUnits with the + // same register as the anti-dependency, don't attempt to + // break it. + for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), + PE = CriticalPathSU->Preds.end(); P != PE; ++P) + if (P->getSUnit() == NextSU ? + (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : + (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { + AntiDepReg = 0; + break; + } + } + } + CriticalPathSU = NextSU; + CriticalPathMI = CriticalPathSU->getInstr(); + } else { + // We've reached the end of the critical path. + CriticalPathSU = 0; + CriticalPathMI = 0; + } + } + + PrescanInstruction(MI); + + if (MI->getDesc().hasExtraDefRegAllocReq()) + // If this instruction's defs have special allocation requirement, don't + // break this anti-dependency. + AntiDepReg = 0; + else if (AntiDepReg) { + // If this instruction has a use of AntiDepReg, breaking it + // is invalid. + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + if (MO.isUse() && AntiDepReg == Reg) { + AntiDepReg = 0; + break; + } + } + } + + // Determine AntiDepReg's register class, if it is live and is + // consistently used within a single class. + const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; + assert((AntiDepReg == 0 || RC != NULL) && + "Register should be live if it's causing an anti-dependence!"); + if (RC == reinterpret_cast(-1)) + AntiDepReg = 0; + + // Look for a suitable register to use to break the anti-depenence. + // + // TODO: Instead of picking the first free register, consider which might + // be the best. + if (AntiDepReg != 0) { + if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, + LastNewReg[AntiDepReg], + RC)) { + DEBUG(errs() << "Breaking anti-dependence edge on " + << TRI->getName(AntiDepReg) + << " with " << RegRefs.count(AntiDepReg) << " references" + << " using " << TRI->getName(NewReg) << "!\n"); + + // Update the references to the old register to refer to the new + // register. + std::pair::iterator, + std::multimap::iterator> + Range = RegRefs.equal_range(AntiDepReg); + for (std::multimap::iterator + Q = Range.first, QE = Range.second; Q != QE; ++Q) + Q->second->setReg(NewReg); + + // We just went back in time and modified history; the + // liveness information for the anti-depenence reg is now + // inconsistent. Set the state as if it were dead. + Classes[NewReg] = Classes[AntiDepReg]; + DefIndices[NewReg] = DefIndices[AntiDepReg]; + KillIndices[NewReg] = KillIndices[AntiDepReg]; + assert(((KillIndices[NewReg] == ~0u) != + (DefIndices[NewReg] == ~0u)) && + "Kill and Def maps aren't consistent for NewReg!"); + + Classes[AntiDepReg] = 0; + DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; + KillIndices[AntiDepReg] = ~0u; + assert(((KillIndices[AntiDepReg] == ~0u) != + (DefIndices[AntiDepReg] == ~0u)) && + "Kill and Def maps aren't consistent for AntiDepReg!"); + + RegRefs.erase(AntiDepReg); + LastNewReg[AntiDepReg] = NewReg; + ++Broken; + } + } + + ScanInstruction(MI, Count); + } + + return Broken; +} Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85127&view=auto ============================================================================== --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (added) +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 11:59:04 2009 @@ -0,0 +1,95 @@ +//=- llvm/CodeGen/CriticalAntiDepBreaker.h - Anti-Dep Support -*- C++ -*-=// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements the CriticalAntiDepBreaker class, which +// implements register anti-dependence breaking along a blocks +// critical path during post-RA scheduler. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H +#define LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H + +#include "llvm/CodeGen/AntiDepBreaker.h" +#include "llvm/CodeGen/MachineBasicBlock.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/CodeGen/MachineFunction.h" +#include "llvm/CodeGen/MachineRegisterInfo.h" +#include "llvm/CodeGen/ScheduleDAG.h" +#include "llvm/Target/TargetRegisterInfo.h" +#include "llvm/ADT/BitVector.h" +#include "llvm/ADT/SmallSet.h" + +namespace llvm { + class CriticalAntiDepBreaker : public AntiDepBreaker { + MachineFunction& MF; + MachineRegisterInfo &MRI; + const TargetRegisterInfo *TRI; + + /// AllocatableSet - The set of allocatable registers. + /// We'll be ignoring anti-dependencies on non-allocatable registers, + /// because they may not be safe to break. + const BitVector AllocatableSet; + + /// Classes - For live regs that are only used in one register class in a + /// live range, the register class. If the register is not live, the + /// corresponding value is null. If the register is live but used in + /// multiple register classes, the corresponding value is -1 casted to a + /// pointer. + const TargetRegisterClass * + Classes[TargetRegisterInfo::FirstVirtualRegister]; + + /// RegRegs - Map registers to all their references within a live range. + std::multimap RegRefs; + + /// KillIndices - The index of the most recent kill (proceding bottom-up), + /// or ~0u if the register is not live. + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; + + /// DefIndices - The index of the most recent complete def (proceding bottom + /// up), or ~0u if the register is live. + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; + + /// KeepRegs - A set of registers which are live and cannot be changed to + /// break anti-dependencies. + SmallSet KeepRegs; + + public: + CriticalAntiDepBreaker(MachineFunction& MFi); + ~CriticalAntiDepBreaker(); + + /// Start - Initialize anti-dep breaking for a new basic block. + void StartBlock(MachineBasicBlock *BB); + + /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path + /// of the ScheduleDAG and break them by renaming registers. + /// + unsigned BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex); + + /// Observe - Update liveness information to account for the current + /// instruction, which will not be scheduled. + /// + void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); + + /// Finish - Finish anti-dep breaking for a basic block. + void FinishBlock(); + + private: + void PrescanInstruction(MachineInstr *MI); + void ScanInstruction(MachineInstr *MI, unsigned Count); + unsigned findSuitableFreeRegister(unsigned AntiDepReg, + unsigned LastNewReg, + const TargetRegisterClass *); + }; +} + +#endif Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85127&r1=85126&r2=85127&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 11:59:04 2009 @@ -19,6 +19,7 @@ //===----------------------------------------------------------------------===// #define DEBUG_TYPE "post-RA-sched" +#include "CriticalAntiDepBreaker.h" #include "ExactHazardRecognizer.h" #include "SimpleHazardRecognizer.h" #include "ScheduleDAGInstrs.h" @@ -40,6 +41,7 @@ #include "llvm/Support/Debug.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/raw_ostream.h" +#include "llvm/ADT/BitVector.h" #include "llvm/ADT/Statistic.h" #include #include @@ -47,6 +49,7 @@ STATISTIC(NumNoops, "Number of noops inserted"); STATISTIC(NumStalls, "Number of pipeline stalls"); +STATISTIC(NumFixedAnti, "Number of fixed anti-dependencies"); // Post-RA scheduling is enabled with // TargetSubtarget.enablePostRAScheduler(). This flag can be used to @@ -55,10 +58,11 @@ EnablePostRAScheduler("post-RA-scheduler", cl::desc("Enable scheduling after register allocation"), cl::init(false), cl::Hidden); -static cl::opt +static cl::opt EnableAntiDepBreaking("break-anti-dependencies", - cl::desc("Break post-RA scheduling anti-dependencies"), - cl::init(true), cl::Hidden); + cl::desc("Break post-RA scheduling anti-dependencies: " + "\"critical\", \"all\", or \"none\""), + cl::init("none"), cl::Hidden); static cl::opt EnablePostRAHazardAvoidance("avoid-hazards", cl::desc("Enable exact hazard avoidance"), @@ -116,56 +120,30 @@ /// Topo - A topological ordering for SUnits. ScheduleDAGTopologicalSort Topo; - /// AllocatableSet - The set of allocatable registers. - /// We'll be ignoring anti-dependencies on non-allocatable registers, - /// because they may not be safe to break. - const BitVector AllocatableSet; - /// HazardRec - The hazard recognizer to use. ScheduleHazardRecognizer *HazardRec; + /// AntiDepBreak - Anti-dependence breaking object, or NULL if none + AntiDepBreaker *AntiDepBreak; + /// AA - AliasAnalysis for making memory reference queries. AliasAnalysis *AA; - /// AntiDepMode - Anti-dependence breaking mode - TargetSubtarget::AntiDepBreakMode AntiDepMode; - - /// Classes - For live regs that are only used in one register class in a - /// live range, the register class. If the register is not live, the - /// corresponding value is null. If the register is live but used in - /// multiple register classes, the corresponding value is -1 casted to a - /// pointer. - const TargetRegisterClass * - Classes[TargetRegisterInfo::FirstVirtualRegister]; - - /// RegRegs - Map registers to all their references within a live range. - std::multimap RegRefs; - /// KillIndices - The index of the most recent kill (proceding bottom-up), /// or ~0u if the register is not live. unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; - /// DefIndices - The index of the most recent complete def (proceding bottom - /// up), or ~0u if the register is live. - unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; - - /// KeepRegs - A set of registers which are live and cannot be changed to - /// break anti-dependencies. - SmallSet KeepRegs; - public: SchedulePostRATDList(MachineFunction &MF, const MachineLoopInfo &MLI, const MachineDominatorTree &MDT, ScheduleHazardRecognizer *HR, - AliasAnalysis *aa, - TargetSubtarget::AntiDepBreakMode adm) + AntiDepBreaker *ADB, + AliasAnalysis *aa) : ScheduleDAGInstrs(MF, MLI, MDT), Topo(SUnits), - AllocatableSet(TRI->getAllocatableSet(MF)), - HazardRec(HR), AA(aa), AntiDepMode(adm) {} + HazardRec(HR), AntiDepBreak(ADB), AA(aa) {} ~SchedulePostRATDList() { - delete HazardRec; } /// StartBlock - Initialize register live-range state for scheduling in @@ -177,11 +155,6 @@ /// void Schedule(); - /// FixupKills - Fix register kill flags that have been made - /// invalid due to scheduling - /// - void FixupKills(MachineBasicBlock *MBB); - /// Observe - Update liveness information to account for the current /// instruction, which will not be scheduled. /// @@ -191,17 +164,16 @@ /// void FinishBlock(); + /// FixupKills - Fix register kill flags that have been made + /// invalid due to scheduling + /// + void FixupKills(MachineBasicBlock *MBB); + private: - void PrescanInstruction(MachineInstr *MI); - void ScanInstruction(MachineInstr *MI, unsigned Count); void ReleaseSucc(SUnit *SU, SDep *SuccEdge); void ReleaseSuccessors(SUnit *SU); void ScheduleNodeTopDown(SUnit *SU, unsigned CurCycle); void ListScheduleTopDown(); - bool BreakAntiDependencies(); - unsigned findSuitableFreeRegister(unsigned AntiDepReg, - unsigned LastNewReg, - const TargetRegisterClass *); void StartBlockForKills(MachineBasicBlock *BB); // ToggleKillFlag - Toggle a register operand kill flag. Other @@ -250,8 +222,9 @@ // Check for antidep breaking override... if (EnableAntiDepBreaking.getPosition() > 0) { - AntiDepMode = (EnableAntiDepBreaking) ? - TargetSubtarget::ANTIDEP_CRITICAL : TargetSubtarget::ANTIDEP_NONE; + AntiDepMode = (EnableAntiDepBreaking == "all") ? TargetSubtarget::ANTIDEP_ALL : + (EnableAntiDepBreaking == "critical") ? TargetSubtarget::ANTIDEP_CRITICAL : + TargetSubtarget::ANTIDEP_NONE; } DEBUG(errs() << "PostRAScheduler\n"); @@ -262,8 +235,12 @@ ScheduleHazardRecognizer *HR = EnablePostRAHazardAvoidance ? (ScheduleHazardRecognizer *)new ExactHazardRecognizer(InstrItins) : (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); + AntiDepBreaker *ADB = + (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME */ : + (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? + new CriticalAntiDepBreaker(Fn) : NULL; - SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, AA, AntiDepMode); + SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); // Loop over all of the basic blocks for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end(); @@ -311,6 +288,9 @@ Scheduler.FixupKills(MBB); } + delete HR; + delete ADB; + return true; } @@ -321,78 +301,10 @@ // Call the superclass. ScheduleDAGInstrs::StartBlock(BB); - // Reset the hazard recognizer. + // Reset the hazard recognizer and anti-dep breaker. HazardRec->Reset(); - - // Clear out the register class data. - std::fill(Classes, array_endof(Classes), - static_cast(0)); - - // Initialize the indices to indicate that no registers are live. - std::fill(KillIndices, array_endof(KillIndices), ~0u); - std::fill(DefIndices, array_endof(DefIndices), BB->size()); - - // Clear "do not change" set. - KeepRegs.clear(); - - bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); - - // Determine the live-out physregs for this block. - if (IsReturnBlock) { - // In a return block, examine the function live-out regs. - for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), - E = MRI.liveout_end(); I != E; ++I) { - unsigned Reg = *I; - Classes[Reg] = reinterpret_cast(-1); - KillIndices[Reg] = BB->size(); - DefIndices[Reg] = ~0u; - // Repeat, for all aliases. - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { - unsigned AliasReg = *Alias; - Classes[AliasReg] = reinterpret_cast(-1); - KillIndices[AliasReg] = BB->size(); - DefIndices[AliasReg] = ~0u; - } - } - } else { - // In a non-return block, examine the live-in regs of all successors. - for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), - SE = BB->succ_end(); SI != SE; ++SI) - for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), - E = (*SI)->livein_end(); I != E; ++I) { - unsigned Reg = *I; - Classes[Reg] = reinterpret_cast(-1); - KillIndices[Reg] = BB->size(); - DefIndices[Reg] = ~0u; - // Repeat, for all aliases. - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { - unsigned AliasReg = *Alias; - Classes[AliasReg] = reinterpret_cast(-1); - KillIndices[AliasReg] = BB->size(); - DefIndices[AliasReg] = ~0u; - } - } - } - - // Mark live-out callee-saved registers. In a return block this is - // all callee-saved registers. In non-return this is any - // callee-saved register that is not saved in the prolog. - const MachineFrameInfo *MFI = MF.getFrameInfo(); - BitVector Pristine = MFI->getPristineRegs(BB); - for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { - unsigned Reg = *I; - if (!IsReturnBlock && !Pristine.test(Reg)) continue; - Classes[Reg] = reinterpret_cast(-1); - KillIndices[Reg] = BB->size(); - DefIndices[Reg] = ~0u; - // Repeat, for all aliases. - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { - unsigned AliasReg = *Alias; - Classes[AliasReg] = reinterpret_cast(-1); - KillIndices[AliasReg] = BB->size(); - DefIndices[AliasReg] = ~0u; - } - } + if (AntiDepBreak != NULL) + AntiDepBreak->StartBlock(BB); } /// Schedule - Schedule the instruction range using list scheduling. @@ -403,8 +315,11 @@ // Build the scheduling graph. BuildSchedGraph(AA); - if (AntiDepMode != TargetSubtarget::ANTIDEP_NONE) { - if (BreakAntiDependencies()) { + if (AntiDepBreak != NULL) { + unsigned Broken = + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, + InsertPosIndex); + if (Broken > 0) { // We made changes. Update the dependency graph. // Theoretically we could update the graph in place: // When a live range is changed to use a different register, remove @@ -415,6 +330,8 @@ EntrySU = SUnit(); ExitSU = SUnit(); BuildSchedGraph(AA); + + NumFixedAnti += Broken; } } @@ -432,436 +349,20 @@ /// instruction, which will not be scheduled. /// void SchedulePostRATDList::Observe(MachineInstr *MI, unsigned Count) { - assert(Count < InsertPosIndex && "Instruction index out of expected range!"); - - // Any register which was defined within the previous scheduling region - // may have been rescheduled and its lifetime may overlap with registers - // in ways not reflected in our current liveness state. For each such - // register, adjust the liveness state to be conservatively correct. - for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) - if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { - assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); - // Mark this register to be non-renamable. - Classes[Reg] = reinterpret_cast(-1); - // Move the def index to the end of the previous region, to reflect - // that the def could theoretically have been scheduled at the end. - DefIndices[Reg] = InsertPosIndex; - } - - PrescanInstruction(MI); - ScanInstruction(MI, Count); + if (AntiDepBreak != NULL) + AntiDepBreak->Observe(MI, Count, InsertPosIndex); } /// FinishBlock - Clean up register live-range state. /// void SchedulePostRATDList::FinishBlock() { - RegRefs.clear(); + if (AntiDepBreak != NULL) + AntiDepBreak->FinishBlock(); // Call the superclass. ScheduleDAGInstrs::FinishBlock(); } -/// CriticalPathStep - Return the next SUnit after SU on the bottom-up -/// critical path. -static SDep *CriticalPathStep(SUnit *SU) { - SDep *Next = 0; - unsigned NextDepth = 0; - // Find the predecessor edge with the greatest depth. - for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); - P != PE; ++P) { - SUnit *PredSU = P->getSUnit(); - unsigned PredLatency = P->getLatency(); - unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; - // In the case of a latency tie, prefer an anti-dependency edge over - // other types of edges. - if (NextDepth < PredTotalLatency || - (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { - NextDepth = PredTotalLatency; - Next = &*P; - } - } - return Next; -} - -void SchedulePostRATDList::PrescanInstruction(MachineInstr *MI) { - // Scan the register operands for this instruction and update - // Classes and RegRefs. - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { - MachineOperand &MO = MI->getOperand(i); - if (!MO.isReg()) continue; - unsigned Reg = MO.getReg(); - if (Reg == 0) continue; - const TargetRegisterClass *NewRC = 0; - - if (i < MI->getDesc().getNumOperands()) - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); - - // For now, only allow the register to be changed if its register - // class is consistent across all uses. - if (!Classes[Reg] && NewRC) - Classes[Reg] = NewRC; - else if (!NewRC || Classes[Reg] != NewRC) - Classes[Reg] = reinterpret_cast(-1); - - // Now check for aliases. - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { - // If an alias of the reg is used during the live range, give up. - // Note that this allows us to skip checking if AntiDepReg - // overlaps with any of the aliases, among other things. - unsigned AliasReg = *Alias; - if (Classes[AliasReg]) { - Classes[AliasReg] = reinterpret_cast(-1); - Classes[Reg] = reinterpret_cast(-1); - } - } - - // If we're still willing to consider this register, note the reference. - if (Classes[Reg] != reinterpret_cast(-1)) - RegRefs.insert(std::make_pair(Reg, &MO)); - - // It's not safe to change register allocation for source operands of - // that have special allocation requirements. - if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { - if (KeepRegs.insert(Reg)) { - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); - *Subreg; ++Subreg) - KeepRegs.insert(*Subreg); - } - } - } -} - -void SchedulePostRATDList::ScanInstruction(MachineInstr *MI, - unsigned Count) { - // Update liveness. - // Proceding upwards, registers that are defed but not used in this - // instruction are now dead. - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { - MachineOperand &MO = MI->getOperand(i); - if (!MO.isReg()) continue; - unsigned Reg = MO.getReg(); - if (Reg == 0) continue; - if (!MO.isDef()) continue; - // Ignore two-addr defs. - if (MI->isRegTiedToUseOperand(i)) continue; - - DefIndices[Reg] = Count; - KillIndices[Reg] = ~0u; - assert(((KillIndices[Reg] == ~0u) != - (DefIndices[Reg] == ~0u)) && - "Kill and Def maps aren't consistent for Reg!"); - KeepRegs.erase(Reg); - Classes[Reg] = 0; - RegRefs.erase(Reg); - // Repeat, for all subregs. - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); - *Subreg; ++Subreg) { - unsigned SubregReg = *Subreg; - DefIndices[SubregReg] = Count; - KillIndices[SubregReg] = ~0u; - KeepRegs.erase(SubregReg); - Classes[SubregReg] = 0; - RegRefs.erase(SubregReg); - } - // Conservatively mark super-registers as unusable. - for (const unsigned *Super = TRI->getSuperRegisters(Reg); - *Super; ++Super) { - unsigned SuperReg = *Super; - Classes[SuperReg] = reinterpret_cast(-1); - } - } - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { - MachineOperand &MO = MI->getOperand(i); - if (!MO.isReg()) continue; - unsigned Reg = MO.getReg(); - if (Reg == 0) continue; - if (!MO.isUse()) continue; - - const TargetRegisterClass *NewRC = 0; - if (i < MI->getDesc().getNumOperands()) - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); - - // For now, only allow the register to be changed if its register - // class is consistent across all uses. - if (!Classes[Reg] && NewRC) - Classes[Reg] = NewRC; - else if (!NewRC || Classes[Reg] != NewRC) - Classes[Reg] = reinterpret_cast(-1); - - RegRefs.insert(std::make_pair(Reg, &MO)); - - // It wasn't previously live but now it is, this is a kill. - if (KillIndices[Reg] == ~0u) { - KillIndices[Reg] = Count; - DefIndices[Reg] = ~0u; - assert(((KillIndices[Reg] == ~0u) != - (DefIndices[Reg] == ~0u)) && - "Kill and Def maps aren't consistent for Reg!"); - } - // Repeat, for all aliases. - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { - unsigned AliasReg = *Alias; - if (KillIndices[AliasReg] == ~0u) { - KillIndices[AliasReg] = Count; - DefIndices[AliasReg] = ~0u; - } - } - } -} - -unsigned -SchedulePostRATDList::findSuitableFreeRegister(unsigned AntiDepReg, - unsigned LastNewReg, - const TargetRegisterClass *RC) { - for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), - RE = RC->allocation_order_end(MF); R != RE; ++R) { - unsigned NewReg = *R; - // Don't replace a register with itself. - if (NewReg == AntiDepReg) continue; - // Don't replace a register with one that was recently used to repair - // an anti-dependence with this AntiDepReg, because that would - // re-introduce that anti-dependence. - if (NewReg == LastNewReg) continue; - // If NewReg is dead and NewReg's most recent def is not before - // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. - assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && - "Kill and Def maps aren't consistent for AntiDepReg!"); - assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && - "Kill and Def maps aren't consistent for NewReg!"); - if (KillIndices[NewReg] != ~0u || - Classes[NewReg] == reinterpret_cast(-1) || - KillIndices[AntiDepReg] > DefIndices[NewReg]) - continue; - return NewReg; - } - - // No registers are free and available! - return 0; -} - -/// BreakAntiDependencies - Identifiy anti-dependencies along the critical path -/// of the ScheduleDAG and break them by renaming registers. -/// -bool SchedulePostRATDList::BreakAntiDependencies() { - // The code below assumes that there is at least one instruction, - // so just duck out immediately if the block is empty. - if (SUnits.empty()) return false; - - // Find the node at the bottom of the critical path. - SUnit *Max = 0; - for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { - SUnit *SU = &SUnits[i]; - if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) - Max = SU; - } - -#ifndef NDEBUG - { - DEBUG(errs() << "Critical path has total latency " - << (Max->getDepth() + Max->Latency) << "\n"); - DEBUG(errs() << "Available regs:"); - for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { - if (KillIndices[Reg] == ~0u) - DEBUG(errs() << " " << TRI->getName(Reg)); - } - DEBUG(errs() << '\n'); - } -#endif - - // Track progress along the critical path through the SUnit graph as we walk - // the instructions. - SUnit *CriticalPathSU = Max; - MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); - - // Consider this pattern: - // A = ... - // ... = A - // A = ... - // ... = A - // A = ... - // ... = A - // A = ... - // ... = A - // There are three anti-dependencies here, and without special care, - // we'd break all of them using the same register: - // A = ... - // ... = A - // B = ... - // ... = B - // B = ... - // ... = B - // B = ... - // ... = B - // because at each anti-dependence, B is the first register that - // isn't A which is free. This re-introduces anti-dependencies - // at all but one of the original anti-dependencies that we were - // trying to break. To avoid this, keep track of the most recent - // register that each register was replaced with, avoid - // using it to repair an anti-dependence on the same register. - // This lets us produce this: - // A = ... - // ... = A - // B = ... - // ... = B - // C = ... - // ... = C - // B = ... - // ... = B - // This still has an anti-dependence on B, but at least it isn't on the - // original critical path. - // - // TODO: If we tracked more than one register here, we could potentially - // fix that remaining critical edge too. This is a little more involved, - // because unlike the most recent register, less recent registers should - // still be considered, though only if no other registers are available. - unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; - - // Attempt to break anti-dependence edges on the critical path. Walk the - // instructions from the bottom up, tracking information about liveness - // as we go to help determine which registers are available. - bool Changed = false; - unsigned Count = InsertPosIndex - 1; - for (MachineBasicBlock::iterator I = InsertPos, E = Begin; - I != E; --Count) { - MachineInstr *MI = --I; - - // Check if this instruction has a dependence on the critical path that - // is an anti-dependence that we may be able to break. If it is, set - // AntiDepReg to the non-zero register associated with the anti-dependence. - // - // We limit our attention to the critical path as a heuristic to avoid - // breaking anti-dependence edges that aren't going to significantly - // impact the overall schedule. There are a limited number of registers - // and we want to save them for the important edges. - // - // TODO: Instructions with multiple defs could have multiple - // anti-dependencies. The current code here only knows how to break one - // edge per instruction. Note that we'd have to be able to break all of - // the anti-dependencies in an instruction in order to be effective. - unsigned AntiDepReg = 0; - if (MI == CriticalPathMI) { - if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { - SUnit *NextSU = Edge->getSUnit(); - - // Only consider anti-dependence edges. - if (Edge->getKind() == SDep::Anti) { - AntiDepReg = Edge->getReg(); - assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); - if (!AllocatableSet.test(AntiDepReg)) - // Don't break anti-dependencies on non-allocatable registers. - AntiDepReg = 0; - else if (KeepRegs.count(AntiDepReg)) - // Don't break anti-dependencies if an use down below requires - // this exact register. - AntiDepReg = 0; - else { - // If the SUnit has other dependencies on the SUnit that it - // anti-depends on, don't bother breaking the anti-dependency - // since those edges would prevent such units from being - // scheduled past each other regardless. - // - // Also, if there are dependencies on other SUnits with the - // same register as the anti-dependency, don't attempt to - // break it. - for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), - PE = CriticalPathSU->Preds.end(); P != PE; ++P) - if (P->getSUnit() == NextSU ? - (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : - (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { - AntiDepReg = 0; - break; - } - } - } - CriticalPathSU = NextSU; - CriticalPathMI = CriticalPathSU->getInstr(); - } else { - // We've reached the end of the critical path. - CriticalPathSU = 0; - CriticalPathMI = 0; - } - } - - PrescanInstruction(MI); - - if (MI->getDesc().hasExtraDefRegAllocReq()) - // If this instruction's defs have special allocation requirement, don't - // break this anti-dependency. - AntiDepReg = 0; - else if (AntiDepReg) { - // If this instruction has a use of AntiDepReg, breaking it - // is invalid. - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { - MachineOperand &MO = MI->getOperand(i); - if (!MO.isReg()) continue; - unsigned Reg = MO.getReg(); - if (Reg == 0) continue; - if (MO.isUse() && AntiDepReg == Reg) { - AntiDepReg = 0; - break; - } - } - } - - // Determine AntiDepReg's register class, if it is live and is - // consistently used within a single class. - const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; - assert((AntiDepReg == 0 || RC != NULL) && - "Register should be live if it's causing an anti-dependence!"); - if (RC == reinterpret_cast(-1)) - AntiDepReg = 0; - - // Look for a suitable register to use to break the anti-depenence. - // - // TODO: Instead of picking the first free register, consider which might - // be the best. - if (AntiDepReg != 0) { - if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, - LastNewReg[AntiDepReg], - RC)) { - DEBUG(errs() << "Breaking anti-dependence edge on " - << TRI->getName(AntiDepReg) - << " with " << RegRefs.count(AntiDepReg) << " references" - << " using " << TRI->getName(NewReg) << "!\n"); - - // Update the references to the old register to refer to the new - // register. - std::pair::iterator, - std::multimap::iterator> - Range = RegRefs.equal_range(AntiDepReg); - for (std::multimap::iterator - Q = Range.first, QE = Range.second; Q != QE; ++Q) - Q->second->setReg(NewReg); - - // We just went back in time and modified history; the - // liveness information for the anti-depenence reg is now - // inconsistent. Set the state as if it were dead. - Classes[NewReg] = Classes[AntiDepReg]; - DefIndices[NewReg] = DefIndices[AntiDepReg]; - KillIndices[NewReg] = KillIndices[AntiDepReg]; - assert(((KillIndices[NewReg] == ~0u) != - (DefIndices[NewReg] == ~0u)) && - "Kill and Def maps aren't consistent for NewReg!"); - - Classes[AntiDepReg] = 0; - DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; - KillIndices[AntiDepReg] = ~0u; - assert(((KillIndices[AntiDepReg] == ~0u) != - (DefIndices[AntiDepReg] == ~0u)) && - "Kill and Def maps aren't consistent for AntiDepReg!"); - - RegRefs.erase(AntiDepReg); - Changed = true; - LastNewReg[AntiDepReg] = NewReg; - } - } - - ScanInstruction(MI, Count); - } - - return Changed; -} - /// StartBlockForKills - Initialize register live-range state for updating kills /// void SchedulePostRATDList::StartBlockForKills(MachineBasicBlock *BB) { Modified: llvm/trunk/lib/Target/ARM/ARMSubtarget.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMSubtarget.h?rev=85127&r1=85126&r2=85127&view=diff ============================================================================== --- llvm/trunk/lib/Target/ARM/ARMSubtarget.h (original) +++ llvm/trunk/lib/Target/ARM/ARMSubtarget.h Mon Oct 26 11:59:04 2009 @@ -130,7 +130,7 @@ /// for Thumb1. bool enablePostRAScheduler(CodeGenOpt::Level OptLevel, TargetSubtarget::AntiDepBreakMode& mode) const { - mode = TargetSubtarget::ANTIDEP_NONE; + mode = TargetSubtarget::ANTIDEP_CRITICAL; return PostRAScheduler && OptLevel >= CodeGenOpt::Default; } Modified: llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll?rev=85127&r1=85126&r2=85127&view=diff ============================================================================== --- llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll (original) +++ llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Mon Oct 26 11:59:04 2009 @@ -1,7 +1,7 @@ -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=false > %t +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=none > %t ; RUN: grep {%xmm0} %t | count 14 ; RUN: not grep {%xmm1} %t -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies > %t +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=critical > %t ; RUN: grep {%xmm0} %t | count 7 ; RUN: grep {%xmm1} %t | count 7 From benny.kra at googlemail.com Mon Oct 26 12:01:22 2009 From: benny.kra at googlemail.com (Benjamin Kramer) Date: Mon, 26 Oct 2009 17:01:22 -0000 Subject: [llvm-commits] [llvm] r85128 - in /llvm/trunk: include/llvm/Support/ include/llvm/System/ lib/Target/Blackfin/ lib/Target/PIC16/PIC16Passes/ utils/count/ utils/not/ utils/unittest/UnitTestMain/ Message-ID: <200910261701.n9QH1Mhu024053@zion.cs.uiuc.edu> Author: d0k Date: Mon Oct 26 12:01:20 2009 New Revision: 85128 URL: http://llvm.org/viewvc/llvm-project?rev=85128&view=rev Log: Some svn:ignore tweaks. Modified: llvm/trunk/include/llvm/Support/ (props changed) llvm/trunk/include/llvm/System/ (props changed) llvm/trunk/lib/Target/Blackfin/ (props changed) llvm/trunk/lib/Target/PIC16/PIC16Passes/ (props changed) llvm/trunk/utils/count/ (props changed) llvm/trunk/utils/not/ (props changed) llvm/trunk/utils/unittest/UnitTestMain/ (props changed) Propchange: llvm/trunk/include/llvm/Support/ ------------------------------------------------------------------------------ --- svn:ignore (original) +++ svn:ignore (removed) @@ -1,2 +0,0 @@ -.cvsignore -DataTypes.h Propchange: llvm/trunk/include/llvm/System/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -0,0 +1 @@ +DataTypes.h Propchange: llvm/trunk/lib/Target/Blackfin/ ------------------------------------------------------------------------------ --- svn:ignore (original) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -6,6 +6,7 @@ BlackfinGenDAGISel.inc BlackfinGenInstrInfo.inc BlackfinGenInstrNames.inc +BlackfinGenIntrinsics.inc BlackfinGenRegisterInfo.h.inc BlackfinGenRegisterInfo.inc BlackfinGenRegisterNames.inc Propchange: llvm/trunk/lib/Target/PIC16/PIC16Passes/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -0,0 +1,4 @@ +Debug +Release-Asserts +Release +Debug+Checks Propchange: llvm/trunk/utils/count/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -0,0 +1,4 @@ +Debug +Release-Asserts +Release +Debug+Checks Propchange: llvm/trunk/utils/not/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -0,0 +1,4 @@ +Debug +Release-Asserts +Release +Debug+Checks Propchange: llvm/trunk/utils/unittest/UnitTestMain/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Oct 26 12:01:20 2009 @@ -0,0 +1,4 @@ +Debug +Release-Asserts +Release +Debug+Checks From dpatel at apple.com Mon Oct 26 12:09:13 2009 From: dpatel at apple.com (Devang Patel) Date: Mon, 26 Oct 2009 17:09:13 -0000 Subject: [llvm-commits] [llvm] r85130 - /llvm/trunk/lib/VMCore/Metadata.cpp Message-ID: <200910261709.n9QH9IIq024599@zion.cs.uiuc.edu> Author: dpatel Date: Mon Oct 26 12:09:00 2009 New Revision: 85130 URL: http://llvm.org/viewvc/llvm-project?rev=85130&view=rev Log: Do not use expensive sort(). Modified: llvm/trunk/lib/VMCore/Metadata.cpp Modified: llvm/trunk/lib/VMCore/Metadata.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/VMCore/Metadata.cpp?rev=85130&r1=85129&r2=85130&view=diff ============================================================================== --- llvm/trunk/lib/VMCore/Metadata.cpp (original) +++ llvm/trunk/lib/VMCore/Metadata.cpp Mon Oct 26 12:09:00 2009 @@ -344,20 +344,22 @@ MDStoreTy::iterator I = MetadataStore.find(Inst); if (I == MetadataStore.end()) return; + MDs.resize(I->second.size()); for (MDMapTy::iterator MI = I->second.begin(), ME = I->second.end(); MI != ME; ++MI) - MDs.push_back(std::make_pair(MI->first, MI->second)); - std::sort(MDs.begin(), MDs.end()); + // MD kinds are numbered from 1. + MDs[MI->first - 1] = std::make_pair(MI->first, MI->second); } /// getHandlerNames - Populate client supplied smallvector using custome /// metadata name and ID. void MetadataContextImpl:: getHandlerNames(SmallVectorImpl >&Names) const { + Names.resize(MDHandlerNames.size()); for (StringMap::const_iterator I = MDHandlerNames.begin(), E = MDHandlerNames.end(); I != E; ++I) - Names.push_back(std::make_pair(I->second, I->first())); - std::sort(Names.begin(), Names.end()); + // MD Handlers are numbered from 1. + Names[I->second - 1] = std::make_pair(I->second, I->first()); } /// ValueIsCloned - This handler is used to update metadata store From sanjiv.gupta at microchip.com Mon Oct 26 13:23:00 2009 From: sanjiv.gupta at microchip.com (Sanjiv Gupta) Date: Mon, 26 Oct 2009 18:23:00 -0000 Subject: [llvm-commits] [llvm] r85134 - in /llvm/trunk/lib/Target/PIC16/PIC16Passes: Makefile PIC16Overlay.cpp Message-ID: <200910261823.n9QIN0hc027720@zion.cs.uiuc.edu> Author: sgupta Date: Mon Oct 26 13:22:59 2009 New Revision: 85134 URL: http://llvm.org/viewvc/llvm-project?rev=85134&view=rev Log: Make PIC16 overlay a loadable pass. Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile?rev=85134&r1=85133&r2=85134&view=diff ============================================================================== --- llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile (original) +++ llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile Mon Oct 26 13:22:59 2009 @@ -8,9 +8,8 @@ ##===----------------------------------------------------------------------===## LEVEL = ../../../.. TARGET = PIC16 -LIBRARYNAME = LLVMpic16passes -BUILD_ARCHIVE = 1 - +LIBRARYNAME = PIC16Passes +LOADABLE_MODULE = 1 include $(LEVEL)/Makefile.common Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp?rev=85134&r1=85133&r2=85134&view=diff ============================================================================== --- llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp (original) +++ llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp Mon Oct 26 13:22:59 2009 @@ -25,7 +25,8 @@ namespace llvm { char PIC16FrameOverlay::ID = 0; - ModulePass *createPIC16OverlayPass() { return new PIC16FrameOverlay(); } + static RegisterPass + X("pic16overlay", "PIC16 Frame Overlay Analysis"); } void PIC16FrameOverlay::getAnalysisUsage(AnalysisUsage &AU) const { From gohman at apple.com Mon Oct 26 13:26:19 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 18:26:19 -0000 Subject: [llvm-commits] [llvm] r85135 - /llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp Message-ID: <200910261826.n9QIQJae027819@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 13:26:18 2009 New Revision: 85135 URL: http://llvm.org/viewvc/llvm-project?rev=85135&view=rev Log: When checking whether a def of an aliased register is dead, ask the machineinstr whether the aliased register is dead, rather than the original register is dead. This allows it to get the correct answer when examining an instruction like this: CALLpcrel32 , %AL, %EAX where EAX is dead but a subregister of it is still live. This fixes PR5294. Modified: llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp Modified: llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp?rev=85135&r1=85134&r2=85135&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp (original) +++ llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp Mon Oct 26 13:26:18 2009 @@ -208,7 +208,7 @@ SUnit *DefSU = DefList[i]; if (DefSU != SU && (Kind != SDep::Output || !MO.isDead() || - !DefSU->getInstr()->registerDefIsDead(Reg))) + !DefSU->getInstr()->registerDefIsDead(*Alias))) DefSU->addPred(SDep(SU, Kind, AOLatency, /*Reg=*/ *Alias)); } } From anton at korobeynikov.info Mon Oct 26 13:33:00 2009 From: anton at korobeynikov.info (Anton Korobeynikov) Date: Mon, 26 Oct 2009 21:33:00 +0300 Subject: [llvm-commits] [llvm] r85134 - in /llvm/trunk/lib/Target/PIC16/PIC16Passes: Makefile PIC16Overlay.cpp In-Reply-To: <200910261823.n9QIN0hc027720@zion.cs.uiuc.edu> References: <200910261823.n9QIN0hc027720@zion.cs.uiuc.edu> Message-ID: Hello, Sanjiv > URL: http://llvm.org/viewvc/llvm-project?rev=85134&view=rev > Log: > Make PIC16 overlay a loadable pass. This commit was reverted previously because it breaks mingw build. It will be reverted again. -- With best regards, Anton Korobeynikov Faculty of Mathematics and Mechanics, Saint Petersburg State University From gohman at apple.com Mon Oct 26 13:36:40 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 18:36:40 -0000 Subject: [llvm-commits] [llvm] r85137 - /llvm/trunk/include/llvm/Support/TargetFolder.h Message-ID: <200910261836.n9QIae0n028340@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 13:36:40 2009 New Revision: 85137 URL: http://llvm.org/viewvc/llvm-project?rev=85137&view=rev Log: Add CreateZExtOrBitCast and CreateSExtOrBitCast to TargetFolder for consistency with ConstantFolder. Modified: llvm/trunk/include/llvm/Support/TargetFolder.h Modified: llvm/trunk/include/llvm/Support/TargetFolder.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/TargetFolder.h?rev=85137&r1=85136&r2=85137&view=diff ============================================================================== --- llvm/trunk/include/llvm/Support/TargetFolder.h (original) +++ llvm/trunk/include/llvm/Support/TargetFolder.h Mon Oct 26 13:36:40 2009 @@ -179,6 +179,16 @@ Constant *CreatePtrToInt(Constant *C, const Type *DestTy) const { return CreateCast(Instruction::PtrToInt, C, DestTy); } + Constant *CreateZExtOrBitCast(Constant *C, const Type *DestTy) const { + if (C->getType() == DestTy) + return C; // avoid calling Fold + return Fold(ConstantExpr::getZExtOrBitCast(C, DestTy)); + } + Constant *CreateSExtOrBitCast(Constant *C, const Type *DestTy) const { + if (C->getType() == DestTy) + return C; // avoid calling Fold + return Fold(ConstantExpr::getSExtOrBitCast(C, DestTy)); + } Constant *CreateTruncOrBitCast(Constant *C, const Type *DestTy) const { if (C->getType() == DestTy) return C; // avoid calling Fold From asl at math.spbu.ru Mon Oct 26 13:40:24 2009 From: asl at math.spbu.ru (Anton Korobeynikov) Date: Mon, 26 Oct 2009 18:40:24 -0000 Subject: [llvm-commits] [llvm] r85138 - in /llvm/trunk/lib/Target/PIC16/PIC16Passes: Makefile PIC16Overlay.cpp Message-ID: <200910261840.n9QIeOew028514@zion.cs.uiuc.edu> Author: asl Date: Mon Oct 26 13:40:24 2009 New Revision: 85138 URL: http://llvm.org/viewvc/llvm-project?rev=85138&view=rev Log: Revert r85134, it breaks mingw build Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile?rev=85138&r1=85137&r2=85138&view=diff ============================================================================== --- llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile (original) +++ llvm/trunk/lib/Target/PIC16/PIC16Passes/Makefile Mon Oct 26 13:40:24 2009 @@ -8,9 +8,8 @@ ##===----------------------------------------------------------------------===## LEVEL = ../../../.. TARGET = PIC16 -LIBRARYNAME = PIC16Passes -LOADABLE_MODULE = 1 - +LIBRARYNAME = LLVMpic16passes +BUILD_ARCHIVE = 1 include $(LEVEL)/Makefile.common Modified: llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp?rev=85138&r1=85137&r2=85138&view=diff ============================================================================== --- llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp (original) +++ llvm/trunk/lib/Target/PIC16/PIC16Passes/PIC16Overlay.cpp Mon Oct 26 13:40:24 2009 @@ -25,8 +25,7 @@ namespace llvm { char PIC16FrameOverlay::ID = 0; - static RegisterPass - X("pic16overlay", "PIC16 Frame Overlay Analysis"); + ModulePass *createPIC16OverlayPass() { return new PIC16FrameOverlay(); } } void PIC16FrameOverlay::getAnalysisUsage(AnalysisUsage &AU) const { From johnny.chen at apple.com Mon Oct 26 13:48:28 2009 From: johnny.chen at apple.com (Johnny Chen) Date: Mon, 26 Oct 2009 11:48:28 -0700 Subject: [llvm-commits] Added more 's' bit set encoding and "rs" register encoding bits Message-ID: <31E018AE-6E13-4514-AF87-D875A8A09C9B@apple.com> Hi, For multiclass AI1_adde_sube_irs definition, I also added: let Inst{31-28} = 0b1110; for the Sri, Srr, and Srs variants. From the AsmString field of the respective record definitions (take SBCSrs, for example): string AsmString = "sbcs $dst, $a, $b"; it looks like this is the right thing to do. Please remove the three "set condition code to ALways" lines from the submitted patch if this is not correct. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: ARMInstrInfo.td.patch Type: application/octet-stream Size: 2981 bytes Desc: not available Url : http://lists.cs.uiuc.edu/pipermail/llvm-commits/attachments/20091026/d73aadb7/attachment.obj From david_goodwin at apple.com Mon Oct 26 14:00:47 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 19:00:47 -0000 Subject: [llvm-commits] [llvm] r85141 - /llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Message-ID: <200910261900.n9QJ0lP6029737@zion.cs.uiuc.edu> Author: david_goodwin Date: Mon Oct 26 14:00:47 2009 New Revision: 85141 URL: http://llvm.org/viewvc/llvm-project?rev=85141&view=rev Log: Add virtual destructor. Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85141&r1=85140&r2=85141&view=diff ============================================================================== --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 14:00:47 2009 @@ -29,6 +29,8 @@ /// anti-dependencies. class AntiDepBreaker { public: + virtual ~AntiDepBreaker() { }; + /// Start - Initialize anti-dep breaking for a new basic block. virtual void StartBlock(MachineBasicBlock *BB) =0; From david_goodwin at apple.com Mon Oct 26 14:02:10 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 12:02:10 -0700 Subject: [llvm-commits] [llvm] r85127 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/CMakeLists.txt lib/CodeGen/CriticalAntiDepBreaker.cpp lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp lib/Target/ARM/ARMSubt In-Reply-To: <74c447500910261138m3326f8a7t859d0bc8eac504e9@mail.gmail.com> References: <74c447500910261138m3326f8a7t859d0bc8eac504e9@mail.gmail.com> Message-ID: Thanks, added virtual destructor.... Author: david_goodwin Date: Mon Oct 26 14:00:47 2009 New Revision: 85141 URL: http://llvm.org/viewvc/llvm-project?rev=85141&view=rev Log: Add virtual destructor. Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h On Oct 26, 2009, at 11:38 AM, Chandler Carruth wrote: > FYI, this change causes both llvm::AntiDepBreaker and > llvm::CriticalAntiDepBreaker to have virtual functions and an > accessible non-virtual destructor. > > On Mon, Oct 26, 2009 at 9:59 AM, David Goodwin > wrote: >> Author: david_goodwin >> Date: Mon Oct 26 11:59:04 2009 >> New Revision: 85127 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=85127&view=rev >> Log: >> Break anti-dependence breaking out into its own class. >> >> Added: >> llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >> Modified: >> llvm/trunk/lib/CodeGen/CMakeLists.txt >> llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >> llvm/trunk/lib/Target/ARM/ARMSubtarget.h >> llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >> >> Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (added) >> +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,56 @@ >> +//=- llvm/CodeGen/AntiDepBreaker.h - Anti-Dependence Breaking -*- C >> ++ -*-=// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the AntiDepBreaker class, which implements >> +// anti-dependence breaking heuristics for post-register- >> allocation scheduling. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#ifndef LLVM_CODEGEN_ANTIDEPBREAKER_H >> +#define LLVM_CODEGEN_ANTIDEPBREAKER_H >> + >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/CodeGen/MachineFunction.h" >> +#include "llvm/CodeGen/MachineRegisterInfo.h" >> +#include "llvm/CodeGen/ScheduleDAG.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> + >> +namespace llvm { >> + >> +/// AntiDepBreaker - This class works into conjunction with the >> +/// post-RA scheduler to rename registers to break register >> +/// anti-dependencies. >> +class AntiDepBreaker { >> +public: >> + /// Start - Initialize anti-dep breaking for a new basic block. >> + virtual void StartBlock(MachineBasicBlock *BB) =0; >> + >> + /// BreakAntiDependencies - Identifiy anti-dependencies within a >> + /// basic-block region and break them by renaming registers. >> Return >> + /// the number of anti-dependencies broken. >> + /// >> + virtual unsigned BreakAntiDependencies(std::vector& SUnits, >> + >> MachineBasicBlock::iterator& Begin, >> + >> MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex) >> =0; >> + >> + /// Observe - Update liveness information to account for the >> current >> + /// instruction, which will not be scheduled. >> + /// >> + virtual void Observe(MachineInstr *MI, unsigned Count, >> + unsigned InsertPosIndex) =0; >> + >> + /// Finish - Finish anti-dep breaking for a basic block. >> + virtual void FinishBlock() =0; >> +}; >> + >> +} >> + >> +#endif >> >> Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) >> +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 11:59:04 2009 >> @@ -1,6 +1,7 @@ >> add_llvm_library(LLVMCodeGen >> BranchFolding.cpp >> CodePlacementOpt.cpp >> + CriticalAntiDepBreaker.cpp >> DeadMachineInstructionElim.cpp >> DwarfEHPrepare.cpp >> ELFCodeEmitter.cpp >> >> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp (added) >> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,539 @@ >> +//===----- CriticalAntiDepBreaker.cpp - Anti-dep breaker -------- >> ---------===// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the CriticalAntiDepBreaker class, which >> +// implements register anti-dependence breaking along a blocks >> +// critical path during post-RA scheduler. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#define DEBUG_TYPE "critical-antidep" >> +#include "CriticalAntiDepBreaker.h" >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/Target/TargetMachine.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> +#include "llvm/Support/Debug.h" >> +#include "llvm/Support/ErrorHandling.h" >> +#include "llvm/Support/raw_ostream.h" >> + >> +using namespace llvm; >> + >> +CriticalAntiDepBreaker:: >> +CriticalAntiDepBreaker(MachineFunction& MFi) : >> + AntiDepBreaker(), MF(MFi), >> + MRI(MF.getRegInfo()), >> + TRI(MF.getTarget().getRegisterInfo()), >> + AllocatableSet(TRI->getAllocatableSet(MF)) >> +{ >> +} >> + >> +CriticalAntiDepBreaker::~CriticalAntiDepBreaker() { >> +} >> + >> +void CriticalAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { >> + // Clear out the register class data. >> + std::fill(Classes, array_endof(Classes), >> + static_cast(0)); >> + >> + // Initialize the indices to indicate that no registers are live. >> + std::fill(KillIndices, array_endof(KillIndices), ~0u); >> + std::fill(DefIndices, array_endof(DefIndices), BB->size()); >> + >> + // Clear "do not change" set. >> + KeepRegs.clear(); >> + >> + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc >> ().isReturn()); >> + >> + // Determine the live-out physregs for this block. >> + if (IsReturnBlock) { >> + // In a return block, examine the function live-out regs. >> + for (MachineRegisterInfo::liveout_iterator I = >> MRI.liveout_begin(), >> + E = MRI.liveout_end(); I != E; ++I) { >> + unsigned Reg = *I; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } else { >> + // In a non-return block, examine the live-in regs of all >> successors. >> + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >> + SE = BB->succ_end(); SI != SE; ++SI) >> + for (MachineBasicBlock::livein_iterator I = (*SI)- >> >livein_begin(), >> + E = (*SI)->livein_end(); I != E; ++I) { >> + unsigned Reg = *I; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); >> *Alias; ++Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast> *>(-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } >> + >> + // Mark live-out callee-saved registers. In a return block this is >> + // all callee-saved registers. In non-return this is any >> + // callee-saved register that is not saved in the prolog. >> + const MachineFrameInfo *MFI = MF.getFrameInfo(); >> + BitVector Pristine = MFI->getPristineRegs(BB); >> + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >> + unsigned Reg = *I; >> + if (!IsReturnBlock && !Pristine.test(Reg)) continue; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> +} >> + >> +void CriticalAntiDepBreaker::FinishBlock() { >> + RegRefs.clear(); >> + KeepRegs.clear(); >> +} >> + >> +void CriticalAntiDepBreaker::Observe(MachineInstr *MI, unsigned >> Count, >> + unsigned InsertPosIndex) { >> + assert(Count < InsertPosIndex && "Instruction index out of >> expected range!"); >> + >> + // Any register which was defined within the previous scheduling >> region >> + // may have been rescheduled and its lifetime may overlap with >> registers >> + // in ways not reflected in our current liveness state. For each >> such >> + // register, adjust the liveness state to be conservatively >> correct. >> + for (unsigned Reg = 0; Reg != >> TargetRegisterInfo::FirstVirtualRegister; ++Reg) >> + if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= >> Count) { >> + assert(KillIndices[Reg] == ~0u && "Clobbered register is >> live!"); >> + // Mark this register to be non-renamable. >> + Classes[Reg] = reinterpret_cast(-1); >> + // Move the def index to the end of the previous region, to >> reflect >> + // that the def could theoretically have been scheduled at >> the end. >> + DefIndices[Reg] = InsertPosIndex; >> + } >> + >> + PrescanInstruction(MI); >> + ScanInstruction(MI, Count); >> +} >> + >> +/// CriticalPathStep - Return the next SUnit after SU on the >> bottom-up >> +/// critical path. >> +static SDep *CriticalPathStep(SUnit *SU) { >> + SDep *Next = 0; >> + unsigned NextDepth = 0; >> + // Find the predecessor edge with the greatest depth. >> + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU- >> >Preds.end(); >> + P != PE; ++P) { >> + SUnit *PredSU = P->getSUnit(); >> + unsigned PredLatency = P->getLatency(); >> + unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >> + // In the case of a latency tie, prefer an anti-dependency >> edge over >> + // other types of edges. >> + if (NextDepth < PredTotalLatency || >> + (NextDepth == PredTotalLatency && P->getKind() == >> SDep::Anti)) { >> + NextDepth = PredTotalLatency; >> + Next = &*P; >> + } >> + } >> + return Next; >> +} >> + >> +void CriticalAntiDepBreaker::PrescanInstruction(MachineInstr *MI) { >> + // Scan the register operands for this instruction and update >> + // Classes and RegRefs. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + const TargetRegisterClass *NewRC = 0; >> + >> + if (i < MI->getDesc().getNumOperands()) >> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> + >> + // For now, only allow the register to be changed if its >> register >> + // class is consistent across all uses. >> + if (!Classes[Reg] && NewRC) >> + Classes[Reg] = NewRC; >> + else if (!NewRC || Classes[Reg] != NewRC) >> + Classes[Reg] = reinterpret_cast(-1); >> + >> + // Now check for aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + // If an alias of the reg is used during the live range, >> give up. >> + // Note that this allows us to skip checking if AntiDepReg >> + // overlaps with any of the aliases, among other things. >> + unsigned AliasReg = *Alias; >> + if (Classes[AliasReg]) { >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + Classes[Reg] = reinterpret_cast(-1); >> + } >> + } >> + >> + // If we're still willing to consider this register, note the >> reference. >> + if (Classes[Reg] != reinterpret_cast(-1)) >> + RegRefs.insert(std::make_pair(Reg, &MO)); >> + >> + // It's not safe to change register allocation for source >> operands of >> + // that have special allocation requirements. >> + if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >> + if (KeepRegs.insert(Reg)) { >> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> + *Subreg; ++Subreg) >> + KeepRegs.insert(*Subreg); >> + } >> + } >> + } >> +} >> + >> +void CriticalAntiDepBreaker::ScanInstruction(MachineInstr *MI, >> + unsigned Count) { >> + // Update liveness. >> + // Proceding upwards, registers that are defed but not used in >> this >> + // instruction are now dead. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (!MO.isDef()) continue; >> + // Ignore two-addr defs. >> + if (MI->isRegTiedToUseOperand(i)) continue; >> + >> + DefIndices[Reg] = Count; >> + KillIndices[Reg] = ~0u; >> + assert(((KillIndices[Reg] == ~0u) != >> + (DefIndices[Reg] == ~0u)) && >> + "Kill and Def maps aren't consistent for Reg!"); >> + KeepRegs.erase(Reg); >> + Classes[Reg] = 0; >> + RegRefs.erase(Reg); >> + // Repeat, for all subregs. >> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> + *Subreg; ++Subreg) { >> + unsigned SubregReg = *Subreg; >> + DefIndices[SubregReg] = Count; >> + KillIndices[SubregReg] = ~0u; >> + KeepRegs.erase(SubregReg); >> + Classes[SubregReg] = 0; >> + RegRefs.erase(SubregReg); >> + } >> + // Conservatively mark super-registers as unusable. >> + for (const unsigned *Super = TRI->getSuperRegisters(Reg); >> + *Super; ++Super) { >> + unsigned SuperReg = *Super; >> + Classes[SuperReg] = reinterpret_cast >> (-1); >> + } >> + } >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (!MO.isUse()) continue; >> + >> + const TargetRegisterClass *NewRC = 0; >> + if (i < MI->getDesc().getNumOperands()) >> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> + >> + // For now, only allow the register to be changed if its >> register >> + // class is consistent across all uses. >> + if (!Classes[Reg] && NewRC) >> + Classes[Reg] = NewRC; >> + else if (!NewRC || Classes[Reg] != NewRC) >> + Classes[Reg] = reinterpret_cast(-1); >> + >> + RegRefs.insert(std::make_pair(Reg, &MO)); >> + >> + // It wasn't previously live but now it is, this is a kill. >> + if (KillIndices[Reg] == ~0u) { >> + KillIndices[Reg] = Count; >> + DefIndices[Reg] = ~0u; >> + assert(((KillIndices[Reg] == ~0u) != >> + (DefIndices[Reg] == ~0u)) && >> + "Kill and Def maps aren't consistent for Reg!"); >> + } >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + if (KillIndices[AliasReg] == ~0u) { >> + KillIndices[AliasReg] = Count; >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } >> +} >> + >> +unsigned >> +CriticalAntiDepBreaker::findSuitableFreeRegister(unsigned >> AntiDepReg, >> + unsigned >> LastNewReg, >> + const >> TargetRegisterClass *RC) { >> + for (TargetRegisterClass::iterator R = RC->allocation_order_begin >> (MF), >> + RE = RC->allocation_order_end(MF); R != RE; ++R) { >> + unsigned NewReg = *R; >> + // Don't replace a register with itself. >> + if (NewReg == AntiDepReg) continue; >> + // Don't replace a register with one that was recently used to >> repair >> + // an anti-dependence with this AntiDepReg, because that would >> + // re-introduce that anti-dependence. >> + if (NewReg == LastNewReg) continue; >> + // If NewReg is dead and NewReg's most recent def is not before >> + // AntiDepReg's kill, it's safe to replace AntiDepReg with >> NewReg. >> + assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices >> [AntiDepReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for AntiDepReg!"); >> + assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == >> ~0u)) && >> + "Kill and Def maps aren't consistent for NewReg!"); >> + if (KillIndices[NewReg] != ~0u || >> + Classes[NewReg] == reinterpret_cast >> (-1) || >> + KillIndices[AntiDepReg] > DefIndices[NewReg]) >> + continue; >> + return NewReg; >> + } >> + >> + // No registers are free and available! >> + return 0; >> +} >> + >> +unsigned CriticalAntiDepBreaker:: >> +BreakAntiDependencies(std::vector& SUnits, >> + MachineBasicBlock::iterator& Begin, >> + MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex) { >> + // The code below assumes that there is at least one instruction, >> + // so just duck out immediately if the block is empty. >> + if (SUnits.empty()) return 0; >> + >> + // Find the node at the bottom of the critical path. >> + SUnit *Max = 0; >> + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >> + SUnit *SU = &SUnits[i]; >> + if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + >> Max->Latency) >> + Max = SU; >> + } >> + >> +#ifndef NDEBUG >> + { >> + DEBUG(errs() << "Critical path has total latency " >> + << (Max->getDepth() + Max->Latency) << "\n"); >> + DEBUG(errs() << "Available regs:"); >> + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >> + if (KillIndices[Reg] == ~0u) >> + DEBUG(errs() << " " << TRI->getName(Reg)); >> + } >> + DEBUG(errs() << '\n'); >> + } >> +#endif >> + >> + // Track progress along the critical path through the SUnit >> graph as we walk >> + // the instructions. >> + SUnit *CriticalPathSU = Max; >> + MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >> + >> + // Consider this pattern: >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // There are three anti-dependencies here, and without special >> care, >> + // we'd break all of them using the same register: >> + // A = ... >> + // ... = A >> + // B = ... >> + // ... = B >> + // B = ... >> + // ... = B >> + // B = ... >> + // ... = B >> + // because at each anti-dependence, B is the first register that >> + // isn't A which is free. This re-introduces anti-dependencies >> + // at all but one of the original anti-dependencies that we were >> + // trying to break. To avoid this, keep track of the most recent >> + // register that each register was replaced with, avoid >> + // using it to repair an anti-dependence on the same register. >> + // This lets us produce this: >> + // A = ... >> + // ... = A >> + // B = ... >> + // ... = B >> + // C = ... >> + // ... = C >> + // B = ... >> + // ... = B >> + // This still has an anti-dependence on B, but at least it isn't >> on the >> + // original critical path. >> + // >> + // TODO: If we tracked more than one register here, we could >> potentially >> + // fix that remaining critical edge too. This is a little more >> involved, >> + // because unlike the most recent register, less recent >> registers should >> + // still be considered, though only if no other registers are >> available. >> + unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = >> {}; >> + >> + // Attempt to break anti-dependence edges on the critical path. >> Walk the >> + // instructions from the bottom up, tracking information about >> liveness >> + // as we go to help determine which registers are available. >> + unsigned Broken = 0; >> + unsigned Count = InsertPosIndex - 1; >> + for (MachineBasicBlock::iterator I = End, E = Begin; >> + I != E; --Count) { >> + MachineInstr *MI = --I; >> + >> + // Check if this instruction has a dependence on the critical >> path that >> + // is an anti-dependence that we may be able to break. If it >> is, set >> + // AntiDepReg to the non-zero register associated with the >> anti-dependence. >> + // >> + // We limit our attention to the critical path as a heuristic >> to avoid >> + // breaking anti-dependence edges that aren't going to >> significantly >> + // impact the overall schedule. There are a limited number of >> registers >> + // and we want to save them for the important edges. >> + // >> + // TODO: Instructions with multiple defs could have multiple >> + // anti-dependencies. The current code here only knows how to >> break one >> + // edge per instruction. Note that we'd have to be able to >> break all of >> + // the anti-dependencies in an instruction in order to be >> effective. >> + unsigned AntiDepReg = 0; >> + if (MI == CriticalPathMI) { >> + if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >> + SUnit *NextSU = Edge->getSUnit(); >> + >> + // Only consider anti-dependence edges. >> + if (Edge->getKind() == SDep::Anti) { >> + AntiDepReg = Edge->getReg(); >> + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >> + if (!AllocatableSet.test(AntiDepReg)) >> + // Don't break anti-dependencies on non-allocatable >> registers. >> + AntiDepReg = 0; >> + else if (KeepRegs.count(AntiDepReg)) >> + // Don't break anti-dependencies if an use down below >> requires >> + // this exact register. >> + AntiDepReg = 0; >> + else { >> + // If the SUnit has other dependencies on the SUnit >> that it >> + // anti-depends on, don't bother breaking the anti- >> dependency >> + // since those edges would prevent such units from being >> + // scheduled past each other regardless. >> + // >> + // Also, if there are dependencies on other SUnits >> with the >> + // same register as the anti-dependency, don't attempt >> to >> + // break it. >> + for (SUnit::pred_iterator P = CriticalPathSU- >> >Preds.begin(), >> + PE = CriticalPathSU->Preds.end(); P != PE; ++P) >> + if (P->getSUnit() == NextSU ? >> + (P->getKind() != SDep::Anti || P->getReg() != >> AntiDepReg) : >> + (P->getKind() == SDep::Data && P->getReg() == >> AntiDepReg)) { >> + AntiDepReg = 0; >> + break; >> + } >> + } >> + } >> + CriticalPathSU = NextSU; >> + CriticalPathMI = CriticalPathSU->getInstr(); >> + } else { >> + // We've reached the end of the critical path. >> + CriticalPathSU = 0; >> + CriticalPathMI = 0; >> + } >> + } >> + >> + PrescanInstruction(MI); >> + >> + if (MI->getDesc().hasExtraDefRegAllocReq()) >> + // If this instruction's defs have special allocation >> requirement, don't >> + // break this anti-dependency. >> + AntiDepReg = 0; >> + else if (AntiDepReg) { >> + // If this instruction has a use of AntiDepReg, breaking it >> + // is invalid. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (MO.isUse() && AntiDepReg == Reg) { >> + AntiDepReg = 0; >> + break; >> + } >> + } >> + } >> + >> + // Determine AntiDepReg's register class, if it is live and is >> + // consistently used within a single class. >> + const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes >> [AntiDepReg] : 0; >> + assert((AntiDepReg == 0 || RC != NULL) && >> + "Register should be live if it's causing an anti- >> dependence!"); >> + if (RC == reinterpret_cast(-1)) >> + AntiDepReg = 0; >> + >> + // Look for a suitable register to use to break the anti- >> depenence. >> + // >> + // TODO: Instead of picking the first free register, consider >> which might >> + // be the best. >> + if (AntiDepReg != 0) { >> + if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >> + LastNewReg >> [AntiDepReg], >> + RC)) { >> + DEBUG(errs() << "Breaking anti-dependence edge on " >> + << TRI->getName(AntiDepReg) >> + << " with " << RegRefs.count(AntiDepReg) << " >> references" >> + << " using " << TRI->getName(NewReg) << "!\n"); >> + >> + // Update the references to the old register to refer to >> the new >> + // register. >> + std::pair> *>::iterator, >> + std::multimap> *>::iterator> >> + Range = RegRefs.equal_range(AntiDepReg); >> + for (std::multimap::iterator >> + Q = Range.first, QE = Range.second; Q != QE; ++Q) >> + Q->second->setReg(NewReg); >> + >> + // We just went back in time and modified history; the >> + // liveness information for the anti-depenence reg is now >> + // inconsistent. Set the state as if it were dead. >> + Classes[NewReg] = Classes[AntiDepReg]; >> + DefIndices[NewReg] = DefIndices[AntiDepReg]; >> + KillIndices[NewReg] = KillIndices[AntiDepReg]; >> + assert(((KillIndices[NewReg] == ~0u) != >> + (DefIndices[NewReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for NewReg!"); >> + >> + Classes[AntiDepReg] = 0; >> + DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >> + KillIndices[AntiDepReg] = ~0u; >> + assert(((KillIndices[AntiDepReg] == ~0u) != >> + (DefIndices[AntiDepReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for AntiDepReg!"); >> + >> + RegRefs.erase(AntiDepReg); >> + LastNewReg[AntiDepReg] = NewReg; >> + ++Broken; >> + } >> + } >> + >> + ScanInstruction(MI, Count); >> + } >> + >> + return Broken; >> +} >> >> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (added) >> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,95 @@ >> +//=- llvm/CodeGen/CriticalAntiDepBreaker.h - Anti-Dep Support -*- C >> ++ -*-=// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the CriticalAntiDepBreaker class, which >> +// implements register anti-dependence breaking along a blocks >> +// critical path during post-RA scheduler. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#ifndef LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >> +#define LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >> + >> +#include "llvm/CodeGen/AntiDepBreaker.h" >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/CodeGen/MachineFunction.h" >> +#include "llvm/CodeGen/MachineRegisterInfo.h" >> +#include "llvm/CodeGen/ScheduleDAG.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> +#include "llvm/ADT/BitVector.h" >> +#include "llvm/ADT/SmallSet.h" >> + >> +namespace llvm { >> + class CriticalAntiDepBreaker : public AntiDepBreaker { >> + MachineFunction& MF; >> + MachineRegisterInfo &MRI; >> + const TargetRegisterInfo *TRI; >> + >> + /// AllocatableSet - The set of allocatable registers. >> + /// We'll be ignoring anti-dependencies on non-allocatable >> registers, >> + /// because they may not be safe to break. >> + const BitVector AllocatableSet; >> + >> + /// Classes - For live regs that are only used in one register >> class in a >> + /// live range, the register class. If the register is not >> live, the >> + /// corresponding value is null. If the register is live but >> used in >> + /// multiple register classes, the corresponding value is -1 >> casted to a >> + /// pointer. >> + const TargetRegisterClass * >> + Classes[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// RegRegs - Map registers to all their references within a >> live range. >> + std::multimap RegRefs; >> + >> + /// KillIndices - The index of the most recent kill (proceding >> bottom-up), >> + /// or ~0u if the register is not live. >> + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// DefIndices - The index of the most recent complete def >> (proceding bottom >> + /// up), or ~0u if the register is live. >> + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// KeepRegs - A set of registers which are live and cannot be >> changed to >> + /// break anti-dependencies. >> + SmallSet KeepRegs; >> + >> + public: >> + CriticalAntiDepBreaker(MachineFunction& MFi); >> + ~CriticalAntiDepBreaker(); >> + >> + /// Start - Initialize anti-dep breaking for a new basic block. >> + void StartBlock(MachineBasicBlock *BB); >> + >> + /// BreakAntiDependencies - Identifiy anti-dependencies along >> the critical path >> + /// of the ScheduleDAG and break them by renaming registers. >> + /// >> + unsigned BreakAntiDependencies(std::vector& SUnits, >> + MachineBasicBlock::iterator& >> Begin, >> + MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex); >> + >> + /// Observe - Update liveness information to account for the >> current >> + /// instruction, which will not be scheduled. >> + /// >> + void Observe(MachineInstr *MI, unsigned Count, unsigned >> InsertPosIndex); >> + >> + /// Finish - Finish anti-dep breaking for a basic block. >> + void FinishBlock(); >> + >> + private: >> + void PrescanInstruction(MachineInstr *MI); >> + void ScanInstruction(MachineInstr *MI, unsigned Count); >> + unsigned findSuitableFreeRegister(unsigned AntiDepReg, >> + unsigned LastNewReg, >> + const TargetRegisterClass *); >> + }; >> +} >> + >> +#endif >> >> Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) >> +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 >> 11:59:04 2009 >> @@ -19,6 +19,7 @@ >> // >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> >> #define DEBUG_TYPE "post-RA-sched" >> +#include "CriticalAntiDepBreaker.h" >> #include "ExactHazardRecognizer.h" >> #include "SimpleHazardRecognizer.h" >> #include "ScheduleDAGInstrs.h" >> @@ -40,6 +41,7 @@ >> #include "llvm/Support/Debug.h" >> #include "llvm/Support/ErrorHandling.h" >> #include "llvm/Support/raw_ostream.h" >> +#include "llvm/ADT/BitVector.h" >> #include "llvm/ADT/Statistic.h" >> #include >> #include >> @@ -47,6 +49,7 @@ >> >> STATISTIC(NumNoops, "Number of noops inserted"); >> STATISTIC(NumStalls, "Number of pipeline stalls"); >> +STATISTIC(NumFixedAnti, "Number of fixed anti-dependencies"); >> >> // Post-RA scheduling is enabled with >> // TargetSubtarget.enablePostRAScheduler(). This flag can be used to >> @@ -55,10 +58,11 @@ >> EnablePostRAScheduler("post-RA-scheduler", >> cl::desc("Enable scheduling after register >> allocation"), >> cl::init(false), cl::Hidden); >> -static cl::opt >> +static cl::opt >> EnableAntiDepBreaking("break-anti-dependencies", >> - cl::desc("Break post-RA scheduling anti- >> dependencies"), >> - cl::init(true), cl::Hidden); >> + cl::desc("Break post-RA scheduling anti- >> dependencies: " >> + "\"critical\", \"all\", or \"none >> \""), >> + cl::init("none"), cl::Hidden); >> static cl::opt >> EnablePostRAHazardAvoidance("avoid-hazards", >> cl::desc("Enable exact hazard avoidance"), >> @@ -116,56 +120,30 @@ >> /// Topo - A topological ordering for SUnits. >> ScheduleDAGTopologicalSort Topo; >> >> - /// AllocatableSet - The set of allocatable registers. >> - /// We'll be ignoring anti-dependencies on non-allocatable >> registers, >> - /// because they may not be safe to break. >> - const BitVector AllocatableSet; >> - >> /// HazardRec - The hazard recognizer to use. >> ScheduleHazardRecognizer *HazardRec; >> >> + /// AntiDepBreak - Anti-dependence breaking object, or NULL if >> none >> + AntiDepBreaker *AntiDepBreak; >> + >> /// AA - AliasAnalysis for making memory reference queries. >> AliasAnalysis *AA; >> >> - /// AntiDepMode - Anti-dependence breaking mode >> - TargetSubtarget::AntiDepBreakMode AntiDepMode; >> - >> - /// Classes - For live regs that are only used in one register >> class in a >> - /// live range, the register class. If the register is not >> live, the >> - /// corresponding value is null. If the register is live but >> used in >> - /// multiple register classes, the corresponding value is -1 >> casted to a >> - /// pointer. >> - const TargetRegisterClass * >> - Classes[TargetRegisterInfo::FirstVirtualRegister]; >> - >> - /// RegRegs - Map registers to all their references within a >> live range. >> - std::multimap RegRefs; >> - >> /// KillIndices - The index of the most recent kill (proceding >> bottom-up), >> /// or ~0u if the register is not live. >> unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >> >> - /// DefIndices - The index of the most recent complete def >> (proceding bottom >> - /// up), or ~0u if the register is live. >> - unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >> - >> - /// KeepRegs - A set of registers which are live and cannot be >> changed to >> - /// break anti-dependencies. >> - SmallSet KeepRegs; >> - >> public: >> SchedulePostRATDList(MachineFunction &MF, >> const MachineLoopInfo &MLI, >> const MachineDominatorTree &MDT, >> ScheduleHazardRecognizer *HR, >> - AliasAnalysis *aa, >> - TargetSubtarget::AntiDepBreakMode adm) >> + AntiDepBreaker *ADB, >> + AliasAnalysis *aa) >> : ScheduleDAGInstrs(MF, MLI, MDT), Topo(SUnits), >> - AllocatableSet(TRI->getAllocatableSet(MF)), >> - HazardRec(HR), AA(aa), AntiDepMode(adm) {} >> + HazardRec(HR), AntiDepBreak(ADB), AA(aa) {} >> >> ~SchedulePostRATDList() { >> - delete HazardRec; >> } >> >> /// StartBlock - Initialize register live-range state for >> scheduling in >> @@ -177,11 +155,6 @@ >> /// >> void Schedule(); >> >> - /// FixupKills - Fix register kill flags that have been made >> - /// invalid due to scheduling >> - /// >> - void FixupKills(MachineBasicBlock *MBB); >> - >> /// Observe - Update liveness information to account for the >> current >> /// instruction, which will not be scheduled. >> /// >> @@ -191,17 +164,16 @@ >> /// >> void FinishBlock(); >> >> + /// FixupKills - Fix register kill flags that have been made >> + /// invalid due to scheduling >> + /// >> + void FixupKills(MachineBasicBlock *MBB); >> + >> private: >> - void PrescanInstruction(MachineInstr *MI); >> - void ScanInstruction(MachineInstr *MI, unsigned Count); >> void ReleaseSucc(SUnit *SU, SDep *SuccEdge); >> void ReleaseSuccessors(SUnit *SU); >> void ScheduleNodeTopDown(SUnit *SU, unsigned CurCycle); >> void ListScheduleTopDown(); >> - bool BreakAntiDependencies(); >> - unsigned findSuitableFreeRegister(unsigned AntiDepReg, >> - unsigned LastNewReg, >> - const TargetRegisterClass *); >> void StartBlockForKills(MachineBasicBlock *BB); >> >> // ToggleKillFlag - Toggle a register operand kill flag. Other >> @@ -250,8 +222,9 @@ >> >> // Check for antidep breaking override... >> if (EnableAntiDepBreaking.getPosition() > 0) { >> - AntiDepMode = (EnableAntiDepBreaking) ? >> - TargetSubtarget::ANTIDEP_CRITICAL : >> TargetSubtarget::ANTIDEP_NONE; >> + AntiDepMode = (EnableAntiDepBreaking == "all") ? >> TargetSubtarget::ANTIDEP_ALL : >> + (EnableAntiDepBreaking == "critical") ? >> TargetSubtarget::ANTIDEP_CRITICAL : >> + TargetSubtarget::ANTIDEP_NONE; >> } >> >> DEBUG(errs() << "PostRAScheduler\n"); >> @@ -262,8 +235,12 @@ >> ScheduleHazardRecognizer *HR = EnablePostRAHazardAvoidance ? >> (ScheduleHazardRecognizer *)new ExactHazardRecognizer >> (InstrItins) : >> (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); >> + AntiDepBreaker *ADB = >> + (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME >> */ : >> + (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? >> + new CriticalAntiDepBreaker(Fn) : NULL; >> >> - SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, AA, AntiDepMode); >> + SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); >> >> // Loop over all of the basic blocks >> for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end(); >> @@ -311,6 +288,9 @@ >> Scheduler.FixupKills(MBB); >> } >> >> + delete HR; >> + delete ADB; >> + >> return true; >> } >> >> @@ -321,78 +301,10 @@ >> // Call the superclass. >> ScheduleDAGInstrs::StartBlock(BB); >> >> - // Reset the hazard recognizer. >> + // Reset the hazard recognizer and anti-dep breaker. >> HazardRec->Reset(); >> - >> - // Clear out the register class data. >> - std::fill(Classes, array_endof(Classes), >> - static_cast(0)); >> - >> - // Initialize the indices to indicate that no registers are live. >> - std::fill(KillIndices, array_endof(KillIndices), ~0u); >> - std::fill(DefIndices, array_endof(DefIndices), BB->size()); >> - >> - // Clear "do not change" set. >> - KeepRegs.clear(); >> - >> - bool IsReturnBlock = (!BB->empty() && BB->back().getDesc >> ().isReturn()); >> - >> - // Determine the live-out physregs for this block. >> - if (IsReturnBlock) { >> - // In a return block, examine the function live-out regs. >> - for (MachineRegisterInfo::liveout_iterator I = >> MRI.liveout_begin(), >> - E = MRI.liveout_end(); I != E; ++I) { >> - unsigned Reg = *I; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } else { >> - // In a non-return block, examine the live-in regs of all >> successors. >> - for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >> - SE = BB->succ_end(); SI != SE; ++SI) >> - for (MachineBasicBlock::livein_iterator I = (*SI)- >> >livein_begin(), >> - E = (*SI)->livein_end(); I != E; ++I) { >> - unsigned Reg = *I; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); >> *Alias; ++Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast> *>(-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } >> - >> - // Mark live-out callee-saved registers. In a return block this is >> - // all callee-saved registers. In non-return this is any >> - // callee-saved register that is not saved in the prolog. >> - const MachineFrameInfo *MFI = MF.getFrameInfo(); >> - BitVector Pristine = MFI->getPristineRegs(BB); >> - for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >> - unsigned Reg = *I; >> - if (!IsReturnBlock && !Pristine.test(Reg)) continue; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->StartBlock(BB); >> } >> >> /// Schedule - Schedule the instruction range using list scheduling. >> @@ -403,8 +315,11 @@ >> // Build the scheduling graph. >> BuildSchedGraph(AA); >> >> - if (AntiDepMode != TargetSubtarget::ANTIDEP_NONE) { >> - if (BreakAntiDependencies()) { >> + if (AntiDepBreak != NULL) { >> + unsigned Broken = >> + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, >> + InsertPosIndex); >> + if (Broken > 0) { >> // We made changes. Update the dependency graph. >> // Theoretically we could update the graph in place: >> // When a live range is changed to use a different register, >> remove >> @@ -415,6 +330,8 @@ >> EntrySU = SUnit(); >> ExitSU = SUnit(); >> BuildSchedGraph(AA); >> + >> + NumFixedAnti += Broken; >> } >> } >> >> @@ -432,436 +349,20 @@ >> /// instruction, which will not be scheduled. >> /// >> void SchedulePostRATDList::Observe(MachineInstr *MI, unsigned >> Count) { >> - assert(Count < InsertPosIndex && "Instruction index out of >> expected range!"); >> - >> - // Any register which was defined within the previous scheduling >> region >> - // may have been rescheduled and its lifetime may overlap with >> registers >> - // in ways not reflected in our current liveness state. For each >> such >> - // register, adjust the liveness state to be conservatively >> correct. >> - for (unsigned Reg = 0; Reg != >> TargetRegisterInfo::FirstVirtualRegister; ++Reg) >> - if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= >> Count) { >> - assert(KillIndices[Reg] == ~0u && "Clobbered register is >> live!"); >> - // Mark this register to be non-renamable. >> - Classes[Reg] = reinterpret_cast(-1); >> - // Move the def index to the end of the previous region, to >> reflect >> - // that the def could theoretically have been scheduled at >> the end. >> - DefIndices[Reg] = InsertPosIndex; >> - } >> - >> - PrescanInstruction(MI); >> - ScanInstruction(MI, Count); >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->Observe(MI, Count, InsertPosIndex); >> } >> >> /// FinishBlock - Clean up register live-range state. >> /// >> void SchedulePostRATDList::FinishBlock() { >> - RegRefs.clear(); >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->FinishBlock(); >> >> // Call the superclass. >> ScheduleDAGInstrs::FinishBlock(); >> } >> >> -/// CriticalPathStep - Return the next SUnit after SU on the >> bottom-up >> -/// critical path. >> -static SDep *CriticalPathStep(SUnit *SU) { >> - SDep *Next = 0; >> - unsigned NextDepth = 0; >> - // Find the predecessor edge with the greatest depth. >> - for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU- >> >Preds.end(); >> - P != PE; ++P) { >> - SUnit *PredSU = P->getSUnit(); >> - unsigned PredLatency = P->getLatency(); >> - unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >> - // In the case of a latency tie, prefer an anti-dependency >> edge over >> - // other types of edges. >> - if (NextDepth < PredTotalLatency || >> - (NextDepth == PredTotalLatency && P->getKind() == >> SDep::Anti)) { >> - NextDepth = PredTotalLatency; >> - Next = &*P; >> - } >> - } >> - return Next; >> -} >> - >> -void SchedulePostRATDList::PrescanInstruction(MachineInstr *MI) { >> - // Scan the register operands for this instruction and update >> - // Classes and RegRefs. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - const TargetRegisterClass *NewRC = 0; >> - >> - if (i < MI->getDesc().getNumOperands()) >> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> - >> - // For now, only allow the register to be changed if its >> register >> - // class is consistent across all uses. >> - if (!Classes[Reg] && NewRC) >> - Classes[Reg] = NewRC; >> - else if (!NewRC || Classes[Reg] != NewRC) >> - Classes[Reg] = reinterpret_cast(-1); >> - >> - // Now check for aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - // If an alias of the reg is used during the live range, >> give up. >> - // Note that this allows us to skip checking if AntiDepReg >> - // overlaps with any of the aliases, among other things. >> - unsigned AliasReg = *Alias; >> - if (Classes[AliasReg]) { >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - Classes[Reg] = reinterpret_cast(-1); >> - } >> - } >> - >> - // If we're still willing to consider this register, note the >> reference. >> - if (Classes[Reg] != reinterpret_cast(-1)) >> - RegRefs.insert(std::make_pair(Reg, &MO)); >> - >> - // It's not safe to change register allocation for source >> operands of >> - // that have special allocation requirements. >> - if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >> - if (KeepRegs.insert(Reg)) { >> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> - *Subreg; ++Subreg) >> - KeepRegs.insert(*Subreg); >> - } >> - } >> - } >> -} >> - >> -void SchedulePostRATDList::ScanInstruction(MachineInstr *MI, >> - unsigned Count) { >> - // Update liveness. >> - // Proceding upwards, registers that are defed but not used in >> this >> - // instruction are now dead. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (!MO.isDef()) continue; >> - // Ignore two-addr defs. >> - if (MI->isRegTiedToUseOperand(i)) continue; >> - >> - DefIndices[Reg] = Count; >> - KillIndices[Reg] = ~0u; >> - assert(((KillIndices[Reg] == ~0u) != >> - (DefIndices[Reg] == ~0u)) && >> - "Kill and Def maps aren't consistent for Reg!"); >> - KeepRegs.erase(Reg); >> - Classes[Reg] = 0; >> - RegRefs.erase(Reg); >> - // Repeat, for all subregs. >> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> - *Subreg; ++Subreg) { >> - unsigned SubregReg = *Subreg; >> - DefIndices[SubregReg] = Count; >> - KillIndices[SubregReg] = ~0u; >> - KeepRegs.erase(SubregReg); >> - Classes[SubregReg] = 0; >> - RegRefs.erase(SubregReg); >> - } >> - // Conservatively mark super-registers as unusable. >> - for (const unsigned *Super = TRI->getSuperRegisters(Reg); >> - *Super; ++Super) { >> - unsigned SuperReg = *Super; >> - Classes[SuperReg] = reinterpret_cast >> (-1); >> - } >> - } >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (!MO.isUse()) continue; >> - >> - const TargetRegisterClass *NewRC = 0; >> - if (i < MI->getDesc().getNumOperands()) >> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> - >> - // For now, only allow the register to be changed if its >> register >> - // class is consistent across all uses. >> - if (!Classes[Reg] && NewRC) >> - Classes[Reg] = NewRC; >> - else if (!NewRC || Classes[Reg] != NewRC) >> - Classes[Reg] = reinterpret_cast(-1); >> - >> - RegRefs.insert(std::make_pair(Reg, &MO)); >> - >> - // It wasn't previously live but now it is, this is a kill. >> - if (KillIndices[Reg] == ~0u) { >> - KillIndices[Reg] = Count; >> - DefIndices[Reg] = ~0u; >> - assert(((KillIndices[Reg] == ~0u) != >> - (DefIndices[Reg] == ~0u)) && >> - "Kill and Def maps aren't consistent for Reg!"); >> - } >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - if (KillIndices[AliasReg] == ~0u) { >> - KillIndices[AliasReg] = Count; >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } >> -} >> - >> -unsigned >> -SchedulePostRATDList::findSuitableFreeRegister(unsigned AntiDepReg, >> - unsigned LastNewReg, >> - const >> TargetRegisterClass *RC) { >> - for (TargetRegisterClass::iterator R = RC->allocation_order_begin >> (MF), >> - RE = RC->allocation_order_end(MF); R != RE; ++R) { >> - unsigned NewReg = *R; >> - // Don't replace a register with itself. >> - if (NewReg == AntiDepReg) continue; >> - // Don't replace a register with one that was recently used to >> repair >> - // an anti-dependence with this AntiDepReg, because that would >> - // re-introduce that anti-dependence. >> - if (NewReg == LastNewReg) continue; >> - // If NewReg is dead and NewReg's most recent def is not before >> - // AntiDepReg's kill, it's safe to replace AntiDepReg with >> NewReg. >> - assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices >> [AntiDepReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for AntiDepReg!"); >> - assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == >> ~0u)) && >> - "Kill and Def maps aren't consistent for NewReg!"); >> - if (KillIndices[NewReg] != ~0u || >> - Classes[NewReg] == reinterpret_cast >> (-1) || >> - KillIndices[AntiDepReg] > DefIndices[NewReg]) >> - continue; >> - return NewReg; >> - } >> - >> - // No registers are free and available! >> - return 0; >> -} >> - >> -/// BreakAntiDependencies - Identifiy anti-dependencies along the >> critical path >> -/// of the ScheduleDAG and break them by renaming registers. >> -/// >> -bool SchedulePostRATDList::BreakAntiDependencies() { >> - // The code below assumes that there is at least one instruction, >> - // so just duck out immediately if the block is empty. >> - if (SUnits.empty()) return false; >> - >> - // Find the node at the bottom of the critical path. >> - SUnit *Max = 0; >> - for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >> - SUnit *SU = &SUnits[i]; >> - if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + >> Max->Latency) >> - Max = SU; >> - } >> - >> -#ifndef NDEBUG >> - { >> - DEBUG(errs() << "Critical path has total latency " >> - << (Max->getDepth() + Max->Latency) << "\n"); >> - DEBUG(errs() << "Available regs:"); >> - for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >> - if (KillIndices[Reg] == ~0u) >> - DEBUG(errs() << " " << TRI->getName(Reg)); >> - } >> - DEBUG(errs() << '\n'); >> - } >> -#endif >> - >> - // Track progress along the critical path through the SUnit >> graph as we walk >> - // the instructions. >> - SUnit *CriticalPathSU = Max; >> - MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >> - >> - // Consider this pattern: >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // There are three anti-dependencies here, and without special >> care, >> - // we'd break all of them using the same register: >> - // A = ... >> - // ... = A >> - // B = ... >> - // ... = B >> - // B = ... >> - // ... = B >> - // B = ... >> - // ... = B >> - // because at each anti-dependence, B is the first register that >> - // isn't A which is free. This re-introduces anti-dependencies >> - // at all but one of the original anti-dependencies that we were >> - // trying to break. To avoid this, keep track of the most recent >> - // register that each register was replaced with, avoid >> - // using it to repair an anti-dependence on the same register. >> - // This lets us produce this: >> - // A = ... >> - // ... = A >> - // B = ... >> - // ... = B >> - // C = ... >> - // ... = C >> - // B = ... >> - // ... = B >> - // This still has an anti-dependence on B, but at least it isn't >> on the >> - // original critical path. >> - // >> - // TODO: If we tracked more than one register here, we could >> potentially >> - // fix that remaining critical edge too. This is a little more >> involved, >> - // because unlike the most recent register, less recent >> registers should >> - // still be considered, though only if no other registers are >> available. >> - unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = >> {}; >> - >> - // Attempt to break anti-dependence edges on the critical path. >> Walk the >> - // instructions from the bottom up, tracking information about >> liveness >> - // as we go to help determine which registers are available. >> - bool Changed = false; >> - unsigned Count = InsertPosIndex - 1; >> - for (MachineBasicBlock::iterator I = InsertPos, E = Begin; >> - I != E; --Count) { >> - MachineInstr *MI = --I; >> - >> - // Check if this instruction has a dependence on the critical >> path that >> - // is an anti-dependence that we may be able to break. If it >> is, set >> - // AntiDepReg to the non-zero register associated with the >> anti-dependence. >> - // >> - // We limit our attention to the critical path as a heuristic >> to avoid >> - // breaking anti-dependence edges that aren't going to >> significantly >> - // impact the overall schedule. There are a limited number of >> registers >> - // and we want to save them for the important edges. >> - // >> - // TODO: Instructions with multiple defs could have multiple >> - // anti-dependencies. The current code here only knows how to >> break one >> - // edge per instruction. Note that we'd have to be able to >> break all of >> - // the anti-dependencies in an instruction in order to be >> effective. >> - unsigned AntiDepReg = 0; >> - if (MI == CriticalPathMI) { >> - if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >> - SUnit *NextSU = Edge->getSUnit(); >> - >> - // Only consider anti-dependence edges. >> - if (Edge->getKind() == SDep::Anti) { >> - AntiDepReg = Edge->getReg(); >> - assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >> - if (!AllocatableSet.test(AntiDepReg)) >> - // Don't break anti-dependencies on non-allocatable >> registers. >> - AntiDepReg = 0; >> - else if (KeepRegs.count(AntiDepReg)) >> - // Don't break anti-dependencies if an use down below >> requires >> - // this exact register. >> - AntiDepReg = 0; >> - else { >> - // If the SUnit has other dependencies on the SUnit >> that it >> - // anti-depends on, don't bother breaking the anti- >> dependency >> - // since those edges would prevent such units from being >> - // scheduled past each other regardless. >> - // >> - // Also, if there are dependencies on other SUnits >> with the >> - // same register as the anti-dependency, don't attempt >> to >> - // break it. >> - for (SUnit::pred_iterator P = CriticalPathSU- >> >Preds.begin(), >> - PE = CriticalPathSU->Preds.end(); P != PE; ++P) >> - if (P->getSUnit() == NextSU ? >> - (P->getKind() != SDep::Anti || P->getReg() != >> AntiDepReg) : >> - (P->getKind() == SDep::Data && P->getReg() == >> AntiDepReg)) { >> - AntiDepReg = 0; >> - break; >> - } >> - } >> - } >> - CriticalPathSU = NextSU; >> - CriticalPathMI = CriticalPathSU->getInstr(); >> - } else { >> - // We've reached the end of the critical path. >> - CriticalPathSU = 0; >> - CriticalPathMI = 0; >> - } >> - } >> - >> - PrescanInstruction(MI); >> - >> - if (MI->getDesc().hasExtraDefRegAllocReq()) >> - // If this instruction's defs have special allocation >> requirement, don't >> - // break this anti-dependency. >> - AntiDepReg = 0; >> - else if (AntiDepReg) { >> - // If this instruction has a use of AntiDepReg, breaking it >> - // is invalid. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (MO.isUse() && AntiDepReg == Reg) { >> - AntiDepReg = 0; >> - break; >> - } >> - } >> - } >> - >> - // Determine AntiDepReg's register class, if it is live and is >> - // consistently used within a single class. >> - const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes >> [AntiDepReg] : 0; >> - assert((AntiDepReg == 0 || RC != NULL) && >> - "Register should be live if it's causing an anti- >> dependence!"); >> - if (RC == reinterpret_cast(-1)) >> - AntiDepReg = 0; >> - >> - // Look for a suitable register to use to break the anti- >> depenence. >> - // >> - // TODO: Instead of picking the first free register, consider >> which might >> - // be the best. >> - if (AntiDepReg != 0) { >> - if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >> - LastNewReg >> [AntiDepReg], >> - RC)) { >> - DEBUG(errs() << "Breaking anti-dependence edge on " >> - << TRI->getName(AntiDepReg) >> - << " with " << RegRefs.count(AntiDepReg) << " >> references" >> - << " using " << TRI->getName(NewReg) << "!\n"); >> - >> - // Update the references to the old register to refer to >> the new >> - // register. >> - std::pair> *>::iterator, >> - std::multimap> *>::iterator> >> - Range = RegRefs.equal_range(AntiDepReg); >> - for (std::multimap::iterator >> - Q = Range.first, QE = Range.second; Q != QE; ++Q) >> - Q->second->setReg(NewReg); >> - >> - // We just went back in time and modified history; the >> - // liveness information for the anti-depenence reg is now >> - // inconsistent. Set the state as if it were dead. >> - Classes[NewReg] = Classes[AntiDepReg]; >> - DefIndices[NewReg] = DefIndices[AntiDepReg]; >> - KillIndices[NewReg] = KillIndices[AntiDepReg]; >> - assert(((KillIndices[NewReg] == ~0u) != >> - (DefIndices[NewReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for NewReg!"); >> - >> - Classes[AntiDepReg] = 0; >> - DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >> - KillIndices[AntiDepReg] = ~0u; >> - assert(((KillIndices[AntiDepReg] == ~0u) != >> - (DefIndices[AntiDepReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for AntiDepReg!"); >> - >> - RegRefs.erase(AntiDepReg); >> - Changed = true; >> - LastNewReg[AntiDepReg] = NewReg; >> - } >> - } >> - >> - ScanInstruction(MI, Count); >> - } >> - >> - return Changed; >> -} >> - >> /// StartBlockForKills - Initialize register live-range state for >> updating kills >> /// >> void SchedulePostRATDList::StartBlockForKills(MachineBasicBlock >> *BB) { >> >> Modified: llvm/trunk/lib/Target/ARM/ARMSubtarget.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMSubtarget.h?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/Target/ARM/ARMSubtarget.h (original) >> +++ llvm/trunk/lib/Target/ARM/ARMSubtarget.h Mon Oct 26 11:59:04 2009 >> @@ -130,7 +130,7 @@ >> /// for Thumb1. >> bool enablePostRAScheduler(CodeGenOpt::Level OptLevel, >> TargetSubtarget::AntiDepBreakMode& >> mode) const { >> - mode = TargetSubtarget::ANTIDEP_NONE; >> + mode = TargetSubtarget::ANTIDEP_CRITICAL; >> return PostRAScheduler && OptLevel >= CodeGenOpt::Default; >> } >> >> >> Modified: llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll (original) >> +++ llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Mon Oct >> 26 11:59:04 2009 >> @@ -1,7 +1,7 @@ >> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=false > %t >> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=none > %t >> ; RUN: grep {%xmm0} %t | count 14 >> ; RUN: not grep {%xmm1} %t >> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies > %t >> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=critical > %t >> ; RUN: grep {%xmm0} %t | count 7 >> ; RUN: grep {%xmm1} %t | count 7 >> >> >> >> _______________________________________________ >> llvm-commits mailing list >> llvm-commits at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits >> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.cs.uiuc.edu/pipermail/llvm-commits/attachments/20091026/7eecfdc4/attachment.html From wendling at apple.com Mon Oct 26 14:07:00 2009 From: wendling at apple.com (Bill Wendling) Date: Mon, 26 Oct 2009 12:07:00 -0700 Subject: [llvm-commits] [llvm] r85141 - /llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h In-Reply-To: <200910261900.n9QJ0lP6029737@zion.cs.uiuc.edu> References: <200910261900.n9QJ0lP6029737@zion.cs.uiuc.edu> Message-ID: On Oct 26, 2009, at 12:00 PM, David Goodwin wrote: > Author: david_goodwin > Date: Mon Oct 26 14:00:47 2009 > New Revision: 85141 > > URL: http://llvm.org/viewvc/llvm-project?rev=85141&view=rev > Log: > Add virtual destructor. > > Modified: > llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h > > Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85141&r1=85140&r2=85141&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) > +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 > 14:00:47 2009 > @@ -29,6 +29,8 @@ > /// anti-dependencies. > class AntiDepBreaker { > public: > + virtual ~AntiDepBreaker() { }; > + It's good to have the definition of a virtual destructor in the .cpp file. It "anchors" the vtable. -bw From gohman at apple.com Mon Oct 26 14:12:14 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 19:12:14 -0000 Subject: [llvm-commits] [llvm] r85144 - in /llvm/trunk: include/llvm/LinkAllPasses.h include/llvm/Transforms/Scalar.h lib/Transforms/Scalar/GEPSplitter.cpp Message-ID: <200910261912.n9QJCE4a030428@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 14:12:14 2009 New Revision: 85144 URL: http://llvm.org/viewvc/llvm-project?rev=85144&view=rev Log: Check in the experimental GEP splitter pass. This pass splits complex GEPs (more than one non-zero index) into simple GEPs (at most one non-zero index). In some simple experiments using this it's not uncommon to see 3% overall code size wins, because it exposes redundancies that can be eliminated, however it's tricky to use because instcombine aggressively undoes the work that this pass does. Added: llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp Modified: llvm/trunk/include/llvm/LinkAllPasses.h llvm/trunk/include/llvm/Transforms/Scalar.h Modified: llvm/trunk/include/llvm/LinkAllPasses.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LinkAllPasses.h?rev=85144&r1=85143&r2=85144&view=diff ============================================================================== --- llvm/trunk/include/llvm/LinkAllPasses.h (original) +++ llvm/trunk/include/llvm/LinkAllPasses.h Mon Oct 26 14:12:14 2009 @@ -141,6 +141,7 @@ (void) llvm::createPartialInliningPass(); (void) llvm::createSSIPass(); (void) llvm::createSSIEverythingPass(); + (void) llvm::createGEPSplitterPass(); (void)new llvm::IntervalPartition(); (void)new llvm::FindUsedTypes(); Modified: llvm/trunk/include/llvm/Transforms/Scalar.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar.h?rev=85144&r1=85143&r2=85144&view=diff ============================================================================== --- llvm/trunk/include/llvm/Transforms/Scalar.h (original) +++ llvm/trunk/include/llvm/Transforms/Scalar.h Mon Oct 26 14:12:14 2009 @@ -341,6 +341,12 @@ // FunctionPass *createSSIEverythingPass(); +//===----------------------------------------------------------------------===// +// +// GEPSplitter - Split complex GEPs into simple ones +// +FunctionPass *createGEPSplitterPass(); + } // End llvm namespace #endif Added: llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp?rev=85144&view=auto ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp (added) +++ llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp Mon Oct 26 14:12:14 2009 @@ -0,0 +1,81 @@ +//===- GEPSplitter.cpp - Split complex GEPs into simple ones --------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This function breaks GEPs with more than 2 non-zero operands into smaller +// GEPs each with no more than 2 non-zero operands. This exposes redundancy +// between GEPs with common initial operand sequences. +// +//===----------------------------------------------------------------------===// + +#define DEBUG_TYPE "split-geps" +#include "llvm/Transforms/Scalar.h" +#include "llvm/Constants.h" +#include "llvm/Function.h" +#include "llvm/Instructions.h" +#include "llvm/Pass.h" +using namespace llvm; + +namespace { + class GEPSplitter : public FunctionPass { + virtual bool runOnFunction(Function &F); + virtual void getAnalysisUsage(AnalysisUsage &AU) const; + public: + static char ID; // Pass identification, replacement for typeid + explicit GEPSplitter() : FunctionPass(&ID) {} + }; +} + +char GEPSplitter::ID = 0; +static RegisterPass X("split-geps", + "split complex GEPs into simple GEPs"); + +FunctionPass *llvm::createGEPSplitterPass() { + return new GEPSplitter(); +} + +bool GEPSplitter::runOnFunction(Function &F) { + bool Changed = false; + + // Visit each GEP instruction. + for (Function::iterator I = F.begin(), E = F.end(); I != E; ++I) + for (BasicBlock::iterator II = I->begin(), IE = I->end(); II != IE; ) + if (GetElementPtrInst *GEP = dyn_cast(II++)) { + unsigned NumOps = GEP->getNumOperands(); + // Ignore GEPs which are already simple. + if (NumOps <= 2) + continue; + bool FirstIndexIsZero = isa(GEP->getOperand(1)) && + cast(GEP->getOperand(1))->isZero(); + if (NumOps == 3 && FirstIndexIsZero) + continue; + // The first index is special and gets expanded with a 2-operand GEP + // (unless it's zero, in which case we can skip this). + Value *NewGEP = FirstIndexIsZero ? + GEP->getOperand(0) : + GetElementPtrInst::Create(GEP->getOperand(0), GEP->getOperand(1), + "tmp", GEP); + // All remaining indices get expanded with a 3-operand GEP with zero + // as the second operand. + Value *Idxs[2]; + Idxs[0] = ConstantInt::get(Type::getInt64Ty(F.getContext()), 0); + for (unsigned i = 2; i != NumOps; ++i) { + Idxs[1] = GEP->getOperand(i); + NewGEP = GetElementPtrInst::Create(NewGEP, Idxs, Idxs+2, "tmp", GEP); + } + GEP->replaceAllUsesWith(NewGEP); + GEP->eraseFromParent(); + Changed = true; + } + + return Changed; +} + +void GEPSplitter::getAnalysisUsage(AnalysisUsage &AU) const { + AU.setPreservesCFG(); +} From jlerouge at apple.com Mon Oct 26 14:30:12 2009 From: jlerouge at apple.com (Julien Lerouge) Date: Mon, 26 Oct 2009 12:30:12 -0700 Subject: [llvm-commits] [llvm] r81719 - in /llvm/trunk: unittests/Makefile unittests/TestMain.cpp utils/unittest/Makefile utils/unittest/UnitTestMain/ utils/unittest/UnitTestMain/Makefile utils/unittest/UnitTestMain/TestMain.cpp In-Reply-To: <20090930215928.GA15682@pom.apple.com> References: <200909132131.n8DLVLIl028742@zion.cs.uiuc.edu> <20090918161136.GB59666@pom.apple.com> <5DDA39F1-474A-4D6D-A039-5B36C6597C5D@apple.com> <20090930215928.GA15682@pom.apple.com> Message-ID: <20091026193011.GA75617@pom.apple.com> On Wed, Sep 30, 2009 at 02:59:29PM -0700, Julien Lerouge wrote: > On Sun, Sep 27, 2009 at 02:20:39PM -0700, Chris Lattner wrote: > > On Sep 18, 2009, at 9:11 AM, Julien Lerouge wrote: > >>> +++ llvm/trunk/utils/unittest/UnitTestMain/Makefile Sun Sep 13 16:31:21 > >>> 2009 > >>> @@ -16,9 +16,4 @@ > >>> CPP.Flags += -I$(LLVM_SRC_ROOT)/utils/unittest/googletest/include > >>> CPP.Flags += -Wno-variadic-macros > >>> > >> > >> -Wno-variadic-macros is not supported with older GCC, can we re-use the > >> logic in googletest/Makefile ? patch attached. > > > > This patch seems reasonable to me, though an autoconf check for those flags > > would be even better (make builds go faster). > > > > This patch adds an autoconf check and removes another flag that was used > in unittests/Makefile.unittest. I have kept the orignal names (NO_xxx). > Should fix the build with older GCC. > > Thanks, > Julien Is it ok to commit this thing ? It's blocking the build of LLVM on MingW with gcc < 4 http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20090928/088200.html Thanks, Julien -- Julien Lerouge PGP Key Id: 0xB1964A62 PGP Fingerprint: 392D 4BAD DB8B CE7F 4E5F FA3C 62DB 4AA7 B196 4A62 PGP Public Key from: keyserver.pgp.com From david_goodwin at apple.com Mon Oct 26 14:32:57 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 19:32:57 -0000 Subject: [llvm-commits] [llvm] r85145 - in /llvm/trunk/lib/CodeGen: AggressiveAntiDepBreaker.cpp AggressiveAntiDepBreaker.h CMakeLists.txt PostRASchedulerList.cpp Message-ID: <200910261933.n9QJX15N031496@zion.cs.uiuc.edu> Author: david_goodwin Date: Mon Oct 26 14:32:42 2009 New Revision: 85145 URL: http://llvm.org/viewvc/llvm-project?rev=85145&view=rev Log: Add aggressive anti-dependence breaker. Currently it is not the default for any target. Enable with -break-anti-dependencies=all. Added: llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Added: llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp?rev=85145&view=auto ============================================================================== --- llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp (added) +++ llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp Mon Oct 26 14:32:42 2009 @@ -0,0 +1,687 @@ +//===----- AggressiveAntiDepBreaker.cpp - Anti-dep breaker -------- ---------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements the AggressiveAntiDepBreaker class, which +// implements register anti-dependence breaking during post-RA +// scheduling. It attempts to break all anti-dependencies within a +// block. +// +//===----------------------------------------------------------------------===// + +#define DEBUG_TYPE "aggressive-antidep" +#include "AggressiveAntiDepBreaker.h" +#include "llvm/CodeGen/MachineBasicBlock.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/CodeGen/MachineInstr.h" +#include "llvm/Target/TargetInstrInfo.h" +#include "llvm/Target/TargetMachine.h" +#include "llvm/Target/TargetRegisterInfo.h" +#include "llvm/Support/Debug.h" +#include "llvm/Support/ErrorHandling.h" +#include "llvm/Support/raw_ostream.h" + +using namespace llvm; + +AggressiveAntiDepBreaker:: +AggressiveAntiDepBreaker(MachineFunction& MFi) : + AntiDepBreaker(), MF(MFi), + MRI(MF.getRegInfo()), + TRI(MF.getTarget().getRegisterInfo()), + AllocatableSet(TRI->getAllocatableSet(MF)), + GroupNodes(TargetRegisterInfo::FirstVirtualRegister, 0) +{ +} + +AggressiveAntiDepBreaker::~AggressiveAntiDepBreaker() { +} + +void AggressiveAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { + // Initialize all registers to be in their own group. Initially we + // assign the register to the same-indexed GroupNode. + for (unsigned i = 0; i < TargetRegisterInfo::FirstVirtualRegister; ++i) + GroupNodeIndices[i] = i; + + // Initialize the indices to indicate that no registers are live. + std::fill(KillIndices, array_endof(KillIndices), ~0u); + std::fill(DefIndices, array_endof(DefIndices), BB->size()); + + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); + + // Determine the live-out physregs for this block. + if (IsReturnBlock) { + // In a return block, examine the function live-out regs. + for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), + E = MRI.liveout_end(); I != E; ++I) { + unsigned Reg = *I; + UnionGroups(Reg, 0); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + UnionGroups(AliasReg, 0); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } + } else { + // In a non-return block, examine the live-in regs of all successors. + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), + SE = BB->succ_end(); SI != SE; ++SI) + for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), + E = (*SI)->livein_end(); I != E; ++I) { + unsigned Reg = *I; + UnionGroups(Reg, 0); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + UnionGroups(AliasReg, 0); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } + } + + // Mark live-out callee-saved registers. In a return block this is + // all callee-saved registers. In non-return this is any + // callee-saved register that is not saved in the prolog. + const MachineFrameInfo *MFI = MF.getFrameInfo(); + BitVector Pristine = MFI->getPristineRegs(BB); + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { + unsigned Reg = *I; + if (!IsReturnBlock && !Pristine.test(Reg)) continue; + UnionGroups(Reg, 0); + KillIndices[Reg] = BB->size(); + DefIndices[Reg] = ~0u; + // Repeat, for all aliases. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + UnionGroups(AliasReg, 0); + KillIndices[AliasReg] = BB->size(); + DefIndices[AliasReg] = ~0u; + } + } +} + +void AggressiveAntiDepBreaker::FinishBlock() { + RegRefs.clear(); +} + +void AggressiveAntiDepBreaker::Observe(MachineInstr *MI, unsigned Count, + unsigned InsertPosIndex) { + assert(Count < InsertPosIndex && "Instruction index out of expected range!"); + + DEBUG(errs() << "Observe: "); + DEBUG(MI->dump()); + + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) { + // If Reg is current live, then mark that it can't be renamed as + // we don't know the extent of its live-range anymore (now that it + // has been scheduled). If it is not live but was defined in the + // previous schedule region, then set its def index to the most + // conservative location (i.e. the beginning of the previous + // schedule region). + if (IsLive(Reg)) { + DEBUG(if (GetGroup(Reg) != 0) + errs() << " " << TRI->getName(Reg) << "=g" << + GetGroup(Reg) << "->g0(region live-out)"); + UnionGroups(Reg, 0); + } else if ((DefIndices[Reg] < InsertPosIndex) && (DefIndices[Reg] >= Count)) { + DefIndices[Reg] = Count; + } + } + + std::set PassthruRegs; + GetPassthruRegs(MI, PassthruRegs); + PrescanInstruction(MI, Count, PassthruRegs); + ScanInstruction(MI, Count); +} + +unsigned AggressiveAntiDepBreaker::GetGroup(unsigned Reg) +{ + unsigned Node = GroupNodeIndices[Reg]; + while (GroupNodes[Node] != Node) + Node = GroupNodes[Node]; + + return Node; +} + +void AggressiveAntiDepBreaker::GetGroupRegs(unsigned Group, std::vector &Regs) +{ + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) { + if (GetGroup(Reg) == Group) + Regs.push_back(Reg); + } +} + +unsigned AggressiveAntiDepBreaker::UnionGroups(unsigned Reg1, unsigned Reg2) +{ + assert(GroupNodes[0] == 0 && "GroupNode 0 not parent!"); + assert(GroupNodeIndices[0] == 0 && "Reg 0 not in Group 0!"); + + // find group for each register + unsigned Group1 = GetGroup(Reg1); + unsigned Group2 = GetGroup(Reg2); + + // if either group is 0, then that must become the parent + unsigned Parent = (Group1 == 0) ? Group1 : Group2; + unsigned Other = (Parent == Group1) ? Group2 : Group1; + GroupNodes.at(Other) = Parent; + return Parent; +} + +unsigned AggressiveAntiDepBreaker::LeaveGroup(unsigned Reg) +{ + // Create a new GroupNode for Reg. Reg's existing GroupNode must + // stay as is because there could be other GroupNodes referring to + // it. + unsigned idx = GroupNodes.size(); + GroupNodes.push_back(idx); + GroupNodeIndices[Reg] = idx; + return idx; +} + +bool AggressiveAntiDepBreaker::IsLive(unsigned Reg) +{ + // KillIndex must be defined and DefIndex not defined for a register + // to be live. + return((KillIndices[Reg] != ~0u) && (DefIndices[Reg] == ~0u)); +} + +bool AggressiveAntiDepBreaker::IsImplicitDefUse(MachineInstr *MI, + MachineOperand& MO) +{ + if (!MO.isReg() || !MO.isImplicit()) + return false; + + unsigned Reg = MO.getReg(); + if (Reg == 0) + return false; + + MachineOperand *Op = NULL; + if (MO.isDef()) + Op = MI->findRegisterUseOperand(Reg, true); + else + Op = MI->findRegisterDefOperand(Reg); + + return((Op != NULL) && Op->isImplicit()); +} + +void AggressiveAntiDepBreaker::GetPassthruRegs(MachineInstr *MI, + std::set& PassthruRegs) { + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + if ((MO.isDef() && MI->isRegTiedToUseOperand(i)) || + IsImplicitDefUse(MI, MO)) { + const unsigned Reg = MO.getReg(); + PassthruRegs.insert(Reg); + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) { + PassthruRegs.insert(*Subreg); + } + } + } +} + +/// AntiDepPathStep - Return SUnit that SU has an anti-dependence on. +static void AntiDepPathStep(SUnit *SU, std::vector& Edges) { + SmallSet Dups; + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); + P != PE; ++P) { + if (P->getKind() == SDep::Anti) { + unsigned Reg = P->getReg(); + if (Dups.count(Reg) == 0) { + Edges.push_back(&*P); + Dups.insert(Reg); + } + } + } +} + +void AggressiveAntiDepBreaker::PrescanInstruction(MachineInstr *MI, unsigned Count, + std::set& PassthruRegs) { + // Scan the register defs for this instruction and update + // live-ranges, groups and RegRefs. + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg() || !MO.isDef()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + // Ignore passthru registers for liveness... + if (PassthruRegs.count(Reg) != 0) continue; + + // Update Def for Reg and subregs. + DefIndices[Reg] = Count; + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) { + unsigned SubregReg = *Subreg; + DefIndices[SubregReg] = Count; + } + } + + DEBUG(errs() << "\tDef Groups:"); + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg() || !MO.isDef()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + + DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << GetGroup(Reg)); + + // If MI's defs have special allocation requirement, don't allow + // any def registers to be changed. Also assume all registers + // defined in a call must not be changed (ABI). + if (MI->getDesc().isCall() || MI->getDesc().hasExtraDefRegAllocReq()) { + DEBUG(if (GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); + UnionGroups(Reg, 0); + } + + // Any aliased that are live at this point are completely or + // partially defined here, so group those subregisters with Reg. + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { + unsigned AliasReg = *Alias; + if (IsLive(AliasReg)) { + UnionGroups(Reg, AliasReg); + DEBUG(errs() << "->g" << GetGroup(Reg) << "(via " << + TRI->getName(AliasReg) << ")"); + } + } + + // Note register reference... + const TargetRegisterClass *RC = NULL; + if (i < MI->getDesc().getNumOperands()) + RC = MI->getDesc().OpInfo[i].getRegClass(TRI); + RegisterReference RR = { &MO, RC }; + RegRefs.insert(std::make_pair(Reg, RR)); + } + + DEBUG(errs() << '\n'); +} + +void AggressiveAntiDepBreaker::ScanInstruction(MachineInstr *MI, + unsigned Count) { + DEBUG(errs() << "\tUse Groups:"); + + // Scan the register uses for this instruction and update + // live-ranges, groups and RegRefs. + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg() || !MO.isUse()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + + DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << GetGroup(Reg)); + + // It wasn't previously live but now it is, this is a kill. Forget + // the previous live-range information and start a new live-range + // for the register. + if (!IsLive(Reg)) { + KillIndices[Reg] = Count; + DefIndices[Reg] = ~0u; + RegRefs.erase(Reg); + LeaveGroup(Reg); + DEBUG(errs() << "->g" << GetGroup(Reg) << "(last-use)"); + } + // Repeat, for subregisters. + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) { + unsigned SubregReg = *Subreg; + if (!IsLive(SubregReg)) { + KillIndices[SubregReg] = Count; + DefIndices[SubregReg] = ~0u; + RegRefs.erase(SubregReg); + LeaveGroup(SubregReg); + DEBUG(errs() << " " << TRI->getName(SubregReg) << "->g" << + GetGroup(SubregReg) << "(last-use)"); + } + } + + // If MI's uses have special allocation requirement, don't allow + // any use registers to be changed. Also assume all registers + // used in a call must not be changed (ABI). + if (MI->getDesc().isCall() || MI->getDesc().hasExtraSrcRegAllocReq()) { + DEBUG(if (GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); + UnionGroups(Reg, 0); + } + + // Note register reference... + const TargetRegisterClass *RC = NULL; + if (i < MI->getDesc().getNumOperands()) + RC = MI->getDesc().OpInfo[i].getRegClass(TRI); + RegisterReference RR = { &MO, RC }; + RegRefs.insert(std::make_pair(Reg, RR)); + } + + DEBUG(errs() << '\n'); + + // Form a group of all defs and uses of a KILL instruction to ensure + // that all registers are renamed as a group. + if (MI->getOpcode() == TargetInstrInfo::KILL) { + DEBUG(errs() << "\tKill Group:"); + + unsigned FirstReg = 0; + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { + MachineOperand &MO = MI->getOperand(i); + if (!MO.isReg()) continue; + unsigned Reg = MO.getReg(); + if (Reg == 0) continue; + + if (FirstReg != 0) { + DEBUG(errs() << "=" << TRI->getName(Reg)); + UnionGroups(FirstReg, Reg); + } else { + DEBUG(errs() << " " << TRI->getName(Reg)); + FirstReg = Reg; + } + } + + DEBUG(errs() << "->g" << GetGroup(FirstReg) << '\n'); + } +} + +BitVector AggressiveAntiDepBreaker::GetRenameRegisters(unsigned Reg) { + BitVector BV(TRI->getNumRegs(), false); + bool first = true; + + // Check all references that need rewriting for Reg. For each, use + // the corresponding register class to narrow the set of registers + // that are appropriate for renaming. + std::pair::iterator, + std::multimap::iterator> + Range = RegRefs.equal_range(Reg); + for (std::multimap::iterator + Q = Range.first, QE = Range.second; Q != QE; ++Q) { + const TargetRegisterClass *RC = Q->second.RC; + if (RC == NULL) continue; + + BitVector RCBV = TRI->getAllocatableSet(MF, RC); + if (first) { + BV |= RCBV; + first = false; + } else { + BV &= RCBV; + } + + DEBUG(errs() << " " << RC->getName()); + } + + return BV; +} + +bool AggressiveAntiDepBreaker::FindSuitableFreeRegisters( + unsigned AntiDepGroupIndex, + std::map &RenameMap) { + // Collect all registers in the same group as AntiDepReg. These all + // need to be renamed together if we are to break the + // anti-dependence. + std::vector Regs; + GetGroupRegs(AntiDepGroupIndex, Regs); + assert(Regs.size() > 0 && "Empty register group!"); + if (Regs.size() == 0) + return false; + + // Find the "superest" register in the group. At the same time, + // collect the BitVector of registers that can be used to rename + // each register. + DEBUG(errs() << "\tRename Candidates for Group g" << AntiDepGroupIndex << ":\n"); + std::map RenameRegisterMap; + unsigned SuperReg = 0; + for (unsigned i = 0, e = Regs.size(); i != e; ++i) { + unsigned Reg = Regs[i]; + if ((SuperReg == 0) || TRI->isSuperRegister(SuperReg, Reg)) + SuperReg = Reg; + + // If Reg has any references, then collect possible rename regs + if (RegRefs.count(Reg) > 0) { + DEBUG(errs() << "\t\t" << TRI->getName(Reg) << ":"); + + BitVector BV = GetRenameRegisters(Reg); + RenameRegisterMap.insert(std::pair(Reg, BV)); + + DEBUG(errs() << " ::"); + DEBUG(for (int r = BV.find_first(); r != -1; r = BV.find_next(r)) + errs() << " " << TRI->getName(r)); + DEBUG(errs() << "\n"); + } + } + + // All group registers should be a subreg of SuperReg. + for (unsigned i = 0, e = Regs.size(); i != e; ++i) { + unsigned Reg = Regs[i]; + if (Reg == SuperReg) continue; + bool IsSub = TRI->isSubRegister(SuperReg, Reg); + assert(IsSub && "Expecting group subregister"); + if (!IsSub) + return false; + } + + // FIXME: for now just handle single register in group case... + if (Regs.size() > 1) + return false; + + // Check each possible rename register for SuperReg. If that register + // is available, and the corresponding registers are available for + // the other group subregisters, then we can use those registers to + // rename. + DEBUG(errs() << "\tFind Register:"); + BitVector SuperBV = RenameRegisterMap[SuperReg]; + for (int r = SuperBV.find_first(); r != -1; r = SuperBV.find_next(r)) { + const unsigned Reg = (unsigned)r; + // Don't replace a register with itself. + if (Reg == SuperReg) continue; + + DEBUG(errs() << " " << TRI->getName(Reg)); + + // If Reg is dead and Reg's most recent def is not before + // SuperRegs's kill, it's safe to replace SuperReg with + // Reg. We must also check all subregisters of Reg. + if (IsLive(Reg) || (KillIndices[SuperReg] > DefIndices[Reg])) { + DEBUG(errs() << "(live)"); + continue; + } else { + bool found = false; + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); + *Subreg; ++Subreg) { + unsigned SubregReg = *Subreg; + if (IsLive(SubregReg) || (KillIndices[SuperReg] > DefIndices[SubregReg])) { + DEBUG(errs() << "(subreg " << TRI->getName(SubregReg) << " live)"); + found = true; + break; + } + } + if (found) + continue; + } + + if (Reg != 0) { + DEBUG(errs() << '\n'); + RenameMap.insert(std::pair(SuperReg, Reg)); + return true; + } + } + + DEBUG(errs() << '\n'); + + // No registers are free and available! + return false; +} + +/// BreakAntiDependencies - Identifiy anti-dependencies within the +/// ScheduleDAG and break them by renaming registers. +/// +unsigned AggressiveAntiDepBreaker::BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex) { + // The code below assumes that there is at least one instruction, + // so just duck out immediately if the block is empty. + if (SUnits.empty()) return false; + + // ...need a map from MI to SUnit. + std::map MISUnitMap; + + DEBUG(errs() << "Breaking all anti-dependencies\n"); + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { + SUnit *SU = &SUnits[i]; + MISUnitMap.insert(std::pair(SU->getInstr(), SU)); + } + +#ifndef NDEBUG + { + DEBUG(errs() << "Available regs:"); + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { + if (!IsLive(Reg)) + DEBUG(errs() << " " << TRI->getName(Reg)); + } + DEBUG(errs() << '\n'); + } +#endif + + // Attempt to break anti-dependence edges. Walk the instructions + // from the bottom up, tracking information about liveness as we go + // to help determine which registers are available. + unsigned Broken = 0; + unsigned Count = InsertPosIndex - 1; + for (MachineBasicBlock::iterator I = End, E = Begin; + I != E; --Count) { + MachineInstr *MI = --I; + + DEBUG(errs() << "Anti: "); + DEBUG(MI->dump()); + + std::set PassthruRegs; + GetPassthruRegs(MI, PassthruRegs); + + // Process the defs in MI... + PrescanInstruction(MI, Count, PassthruRegs); + + std::vector Edges; + SUnit *PathSU = MISUnitMap[MI]; + if (PathSU) + AntiDepPathStep(PathSU, Edges); + + // Ignore KILL instructions (they form a group in ScanInstruction + // but don't cause any anti-dependence breaking themselves) + if (MI->getOpcode() != TargetInstrInfo::KILL) { + // Attempt to break each anti-dependency... + for (unsigned i = 0, e = Edges.size(); i != e; ++i) { + SDep *Edge = Edges[i]; + SUnit *NextSU = Edge->getSUnit(); + + if (Edge->getKind() != SDep::Anti) continue; + + unsigned AntiDepReg = Edge->getReg(); + DEBUG(errs() << "\tAntidep reg: " << TRI->getName(AntiDepReg)); + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); + + if (!AllocatableSet.test(AntiDepReg)) { + // Don't break anti-dependencies on non-allocatable registers. + DEBUG(errs() << " (non-allocatable)\n"); + continue; + } else if (PassthruRegs.count(AntiDepReg) != 0) { + // If the anti-dep register liveness "passes-thru", then + // don't try to change it. It will be changed along with + // the use if required to break an earlier antidep. + DEBUG(errs() << " (passthru)\n"); + continue; + } else { + // No anti-dep breaking for implicit deps + MachineOperand *AntiDepOp = MI->findRegisterDefOperand(AntiDepReg); + assert(AntiDepOp != NULL && "Can't find index for defined register operand"); + if ((AntiDepOp == NULL) || AntiDepOp->isImplicit()) { + DEBUG(errs() << " (implicit)\n"); + continue; + } + + // If the SUnit has other dependencies on the SUnit that + // it anti-depends on, don't bother breaking the + // anti-dependency since those edges would prevent such + // units from being scheduled past each other + // regardless. + for (SUnit::pred_iterator P = PathSU->Preds.begin(), + PE = PathSU->Preds.end(); P != PE; ++P) { + if ((P->getSUnit() == NextSU) && (P->getKind() != SDep::Anti)) { + DEBUG(errs() << " (real dependency)\n"); + AntiDepReg = 0; + break; + } + } + + if (AntiDepReg == 0) continue; + } + + assert(AntiDepReg != 0); + if (AntiDepReg == 0) continue; + + // Determine AntiDepReg's register group. + const unsigned GroupIndex = GetGroup(AntiDepReg); + if (GroupIndex == 0) { + DEBUG(errs() << " (zero group)\n"); + continue; + } + + DEBUG(errs() << '\n'); + + // Look for a suitable register to use to break the anti-dependence. + std::map RenameMap; + if (FindSuitableFreeRegisters(GroupIndex, RenameMap)) { + DEBUG(errs() << "\tBreaking anti-dependence edge on " + << TRI->getName(AntiDepReg) << ":"); + + // Handle each group register... + for (std::map::iterator + S = RenameMap.begin(), E = RenameMap.end(); S != E; ++S) { + unsigned CurrReg = S->first; + unsigned NewReg = S->second; + + DEBUG(errs() << " " << TRI->getName(CurrReg) << "->" << + TRI->getName(NewReg) << "(" << + RegRefs.count(CurrReg) << " refs)"); + + // Update the references to the old register CurrReg to + // refer to the new register NewReg. + std::pair::iterator, + std::multimap::iterator> + Range = RegRefs.equal_range(CurrReg); + for (std::multimap::iterator + Q = Range.first, QE = Range.second; Q != QE; ++Q) { + Q->second.Operand->setReg(NewReg); + } + + // We just went back in time and modified history; the + // liveness information for CurrReg is now inconsistent. Set + // the state as if it were dead. + UnionGroups(NewReg, 0); + RegRefs.erase(NewReg); + DefIndices[NewReg] = DefIndices[CurrReg]; + KillIndices[NewReg] = KillIndices[CurrReg]; + + UnionGroups(CurrReg, 0); + RegRefs.erase(CurrReg); + DefIndices[CurrReg] = KillIndices[CurrReg]; + KillIndices[CurrReg] = ~0u; + assert(((KillIndices[CurrReg] == ~0u) != + (DefIndices[CurrReg] == ~0u)) && + "Kill and Def maps aren't consistent for AntiDepReg!"); + } + + ++Broken; + DEBUG(errs() << '\n'); + } + } + } + + ScanInstruction(MI, Count); + } + + return Broken; +} Added: llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h?rev=85145&view=auto ============================================================================== --- llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h (added) +++ llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h Mon Oct 26 14:32:42 2009 @@ -0,0 +1,136 @@ +//=- llvm/CodeGen/AggressiveAntiDepBreaker.h - Anti-Dep Support -*- C++ -*-=// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements the AggressiveAntiDepBreaker class, which +// implements register anti-dependence breaking during post-RA +// scheduling. It attempts to break all anti-dependencies within a +// block. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_CODEGEN_AGGRESSIVEANTIDEPBREAKER_H +#define LLVM_CODEGEN_AGGRESSIVEANTIDEPBREAKER_H + +#include "llvm/CodeGen/AntiDepBreaker.h" +#include "llvm/CodeGen/MachineBasicBlock.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/CodeGen/MachineFunction.h" +#include "llvm/CodeGen/MachineRegisterInfo.h" +#include "llvm/CodeGen/ScheduleDAG.h" +#include "llvm/Target/TargetRegisterInfo.h" +#include "llvm/ADT/BitVector.h" +#include "llvm/ADT/SmallSet.h" + +namespace llvm { + class AggressiveAntiDepBreaker : public AntiDepBreaker { + MachineFunction& MF; + MachineRegisterInfo &MRI; + const TargetRegisterInfo *TRI; + + /// RegisterReference - Information about a register reference + /// within a liverange + typedef struct { + /// Operand - The registers operand + MachineOperand *Operand; + /// RC - The register class + const TargetRegisterClass *RC; + } RegisterReference; + + /// AllocatableSet - The set of allocatable registers. + /// We'll be ignoring anti-dependencies on non-allocatable registers, + /// because they may not be safe to break. + const BitVector AllocatableSet; + + /// GroupNodes - Implements a disjoint-union data structure to + /// form register groups. A node is represented by an index into + /// the vector. A node can "point to" itself to indicate that it + /// is the parent of a group, or point to another node to indicate + /// that it is a member of the same group as that node. + std::vector GroupNodes; + + /// GroupNodeIndices - For each register, the index of the GroupNode + /// currently representing the group that the register belongs to. + /// Register 0 is always represented by the 0 group, a group + /// composed of registers that are not eligible for anti-aliasing. + unsigned GroupNodeIndices[TargetRegisterInfo::FirstVirtualRegister]; + + /// RegRegs - Map registers to all their references within a live range. + std::multimap RegRefs; + + /// KillIndices - The index of the most recent kill (proceding bottom-up), + /// or ~0u if the register is not live. + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; + + /// DefIndices - The index of the most recent complete def (proceding bottom + /// up), or ~0u if the register is live. + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; + + public: + AggressiveAntiDepBreaker(MachineFunction& MFi); + ~AggressiveAntiDepBreaker(); + + /// Start - Initialize anti-dep breaking for a new basic block. + void StartBlock(MachineBasicBlock *BB); + + /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path + /// of the ScheduleDAG and break them by renaming registers. + /// + unsigned BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex); + + /// Observe - Update liveness information to account for the current + /// instruction, which will not be scheduled. + /// + void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); + + /// Finish - Finish anti-dep breaking for a basic block. + void FinishBlock(); + + private: + // GetGroup - Get the group for a register. The returned value is + // the index of the GroupNode representing the group. + unsigned GetGroup(unsigned Reg); + + // GetGroupRegs - Return a vector of the registers belonging to a + // group. + void GetGroupRegs(unsigned Group, std::vector &Regs); + + // UnionGroups - Union Reg1's and Reg2's groups to form a new + // group. Return the index of the GroupNode representing the + // group. + unsigned UnionGroups(unsigned Reg1, unsigned Reg2); + + // LeaveGroup - Remove a register from its current group and place + // it alone in its own group. Return the index of the GroupNode + // representing the registers new group. + unsigned LeaveGroup(unsigned Reg); + + /// IsLive - Return true if Reg is live + bool IsLive(unsigned Reg); + + /// IsImplicitDefUse - Return true if MO represents a register + /// that is both implicitly used and defined in MI + bool IsImplicitDefUse(MachineInstr *MI, MachineOperand& MO); + + /// GetPassthruRegs - If MI implicitly def/uses a register, then + /// return that register and all subregisters. + void GetPassthruRegs(MachineInstr *MI, std::set& PassthruRegs); + + void PrescanInstruction(MachineInstr *MI, unsigned Count, + std::set& PassthruRegs); + void ScanInstruction(MachineInstr *MI, unsigned Count); + BitVector GetRenameRegisters(unsigned Reg); + bool FindSuitableFreeRegisters(unsigned AntiDepGroupIndex, + std::map &RenameMap); + }; +} + +#endif Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85145&r1=85144&r2=85145&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 14:32:42 2009 @@ -1,4 +1,5 @@ add_llvm_library(LLVMCodeGen + AggressiveAntiDepBreaker.cpp BranchFolding.cpp CodePlacementOpt.cpp CriticalAntiDepBreaker.cpp Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85145&r1=85144&r2=85145&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 14:32:42 2009 @@ -19,6 +19,7 @@ //===----------------------------------------------------------------------===// #define DEBUG_TYPE "post-RA-sched" +#include "AggressiveAntiDepBreaker.h" #include "CriticalAntiDepBreaker.h" #include "ExactHazardRecognizer.h" #include "SimpleHazardRecognizer.h" @@ -236,9 +237,10 @@ (ScheduleHazardRecognizer *)new ExactHazardRecognizer(InstrItins) : (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); AntiDepBreaker *ADB = - (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME */ : - (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? - new CriticalAntiDepBreaker(Fn) : NULL; + ((AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? + (AntiDepBreaker *)new AggressiveAntiDepBreaker(Fn) : + ((AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? + (AntiDepBreaker *)new CriticalAntiDepBreaker(Fn) : NULL)); SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); From anton at korobeynikov.info Mon Oct 26 14:39:40 2009 From: anton at korobeynikov.info (Anton Korobeynikov) Date: Mon, 26 Oct 2009 22:39:40 +0300 Subject: [llvm-commits] [llvm] r81719 - in /llvm/trunk: unittests/Makefile unittests/TestMain.cpp utils/unittest/Makefile utils/unittest/UnitTestMain/ utils/unittest/UnitTestMain/Makefile utils/unittest/UnitTestMain/TestMain.cpp In-Reply-To: <20091026193011.GA75617@pom.apple.com> References: <200909132131.n8DLVLIl028742@zion.cs.uiuc.edu> <20090918161136.GB59666@pom.apple.com> <5DDA39F1-474A-4D6D-A039-5B36C6597C5D@apple.com> <20090930215928.GA15682@pom.apple.com> <20091026193011.GA75617@pom.apple.com> Message-ID: Hello, Julien > Is it ok to commit this thing ? It's blocking the build of LLVM on MingW > with gcc < 4 > > http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20090928/088200.html Looks ok to me, please commit -- With best regards, Anton Korobeynikov Faculty of Mathematics and Mechanics, Saint Petersburg State University From david_goodwin at apple.com Mon Oct 26 14:41:01 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 19:41:01 -0000 Subject: [llvm-commits] [llvm] r85146 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp Message-ID: <200910261941.n9QJf1r5031824@zion.cs.uiuc.edu> Author: david_goodwin Date: Mon Oct 26 14:41:00 2009 New Revision: 85146 URL: http://llvm.org/viewvc/llvm-project?rev=85146&view=rev Log: Define virtual destructor in *.cpp file. Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85146&r1=85145&r2=85146&view=diff ============================================================================== --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 14:41:00 2009 @@ -29,7 +29,7 @@ /// anti-dependencies. class AntiDepBreaker { public: - virtual ~AntiDepBreaker() { }; + virtual ~AntiDepBreaker(); /// Start - Initialize anti-dep breaking for a new basic block. virtual void StartBlock(MachineBasicBlock *BB) =0; Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85146&r1=85145&r2=85146&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 14:41:00 2009 @@ -24,6 +24,7 @@ #include "ExactHazardRecognizer.h" #include "SimpleHazardRecognizer.h" #include "ScheduleDAGInstrs.h" +#include "llvm/CodeGen/AntiDepBreaker.h" #include "llvm/CodeGen/Passes.h" #include "llvm/CodeGen/LatencyPriorityQueue.h" #include "llvm/CodeGen/SchedulerRegistry.h" @@ -79,6 +80,8 @@ cl::desc("Debug control MBBs that are scheduled"), cl::init(0), cl::Hidden); +AntiDepBreaker::~AntiDepBreaker() { } + namespace { class PostRAScheduler : public MachineFunctionPass { AliasAnalysis *AA; From david_goodwin at apple.com Mon Oct 26 14:41:38 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 12:41:38 -0700 Subject: [llvm-commits] [llvm] r85141 - /llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h In-Reply-To: References: <200910261900.n9QJ0lP6029737@zion.cs.uiuc.edu> Message-ID: Sending include/llvm/CodeGen/AntiDepBreaker.h Sending lib/CodeGen/PostRASchedulerList.cpp Transmitting file data .. Committed revision 85146. On Oct 26, 2009, at 12:07 PM, Bill Wendling wrote: > On Oct 26, 2009, at 12:00 PM, David Goodwin wrote: > >> Author: david_goodwin >> Date: Mon Oct 26 14:00:47 2009 >> New Revision: 85141 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=85141&view=rev >> Log: >> Add virtual destructor. >> >> Modified: >> llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> >> Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85141&r1=85140&r2=85141&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) >> +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 >> 14:00:47 2009 >> @@ -29,6 +29,8 @@ >> /// anti-dependencies. >> class AntiDepBreaker { >> public: >> + virtual ~AntiDepBreaker() { }; >> + > > It's good to have the definition of a virtual destructor in the .cpp > file. It "anchors" the vtable. > > -bw > From jlerouge at apple.com Mon Oct 26 14:58:45 2009 From: jlerouge at apple.com (Julien Lerouge) Date: Mon, 26 Oct 2009 19:58:45 -0000 Subject: [llvm-commits] [llvm] r85147 - in /llvm/trunk: Makefile.config.in autoconf/configure.ac autoconf/m4/cxx_flag_check.m4 Message-ID: <200910261958.n9QJwjjp000406@zion.cs.uiuc.edu> Author: jlerouge Date: Mon Oct 26 14:58:44 2009 New Revision: 85147 URL: http://llvm.org/viewvc/llvm-project?rev=85147&view=rev Log: Add an autoconf test to check for optional compiler flags like -Wno-missing-field-initializers or -Wno-variadic-macros. Added: llvm/trunk/autoconf/m4/cxx_flag_check.m4 Modified: llvm/trunk/Makefile.config.in llvm/trunk/autoconf/configure.ac Modified: llvm/trunk/Makefile.config.in URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/Makefile.config.in?rev=85147&r1=85146&r2=85147&view=diff ============================================================================== --- llvm/trunk/Makefile.config.in (original) +++ llvm/trunk/Makefile.config.in Mon Oct 26 14:58:44 2009 @@ -320,3 +320,9 @@ # support (via the -load option). ENABLE_LLVMC_DYNAMIC_PLUGINS = 1 #@ENABLE_LLVMC_DYNAMIC_PLUGINS@ + +# Optional flags supported by the compiler +# -Wno-missing-field-initializers +NO_MISSING_FIELD_INITIALIZERS = @NO_MISSING_FIELD_INITIALIZERS@ +# -Wno-variadic-macros +NO_VARIADIC_MACROS = @NO_VARIADIC_MACROS@ Modified: llvm/trunk/autoconf/configure.ac URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/autoconf/configure.ac?rev=85147&r1=85146&r2=85147&view=diff ============================================================================== --- llvm/trunk/autoconf/configure.ac (original) +++ llvm/trunk/autoconf/configure.ac Mon Oct 26 14:58:44 2009 @@ -943,6 +943,12 @@ dnl Tool compatibility is okay if we make it here. AC_MSG_RESULT([ok]) +dnl Check optional compiler flags. +AC_MSG_CHECKING([optional compiler flags]) +CXX_FLAG_CHECK(NO_VARIADIC_MACROS, [-Wno-variadic-macros]) +CXX_FLAG_CHECK(NO_MISSING_FIELD_INITIALIZERS, [-Wno-missing-field-initializers]) +AC_MSG_RESULT([$NO_VARIADIC_MACROS $NO_MISSING_FIELD_INITIALIZERS]) + dnl===-----------------------------------------------------------------------=== dnl=== dnl=== SECTION 5: Check for libraries Added: llvm/trunk/autoconf/m4/cxx_flag_check.m4 URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/autoconf/m4/cxx_flag_check.m4?rev=85147&view=auto ============================================================================== --- llvm/trunk/autoconf/m4/cxx_flag_check.m4 (added) +++ llvm/trunk/autoconf/m4/cxx_flag_check.m4 Mon Oct 26 14:58:44 2009 @@ -0,0 +1,2 @@ +AC_DEFUN([CXX_FLAG_CHECK], + [AC_SUBST($1, `$CXX $2 -fsyntax-only -xc /dev/null 2>/dev/null && echo $2`)]) From jlerouge at apple.com Mon Oct 26 15:00:35 2009 From: jlerouge at apple.com (Julien Lerouge) Date: Mon, 26 Oct 2009 20:00:35 -0000 Subject: [llvm-commits] [llvm] r85148 - /llvm/trunk/configure Message-ID: <200910262000.n9QK0aDq000551@zion.cs.uiuc.edu> Author: jlerouge Date: Mon Oct 26 15:00:35 2009 New Revision: 85148 URL: http://llvm.org/viewvc/llvm-project?rev=85148&view=rev Log: Regenerate. Modified: llvm/trunk/configure Modified: llvm/trunk/configure URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/configure?rev=85148&r1=85147&r2=85148&view=diff ============================================================================== --- llvm/trunk/configure (original) +++ llvm/trunk/configure Mon Oct 26 15:00:35 2009 @@ -913,6 +913,8 @@ LLVMGXXCOMMAND LLVMGCC LLVMGXX +NO_VARIADIC_MACROS +NO_MISSING_FIELD_INITIALIZERS USE_UDIS86 USE_OPROFILE HAVE_PTHREAD @@ -11008,7 +11010,7 @@ lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext < conftest.$ac_ext + echo '#line 13157 "configure"' > conftest.$ac_ext if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 (eval $ac_compile) 2>&5 ac_status=$? @@ -14870,11 +14872,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:14873: $lt_compile\"" >&5) + (eval echo "\"\$as_me:14875: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:14877: \$? = $ac_status" >&5 + echo "$as_me:14879: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -15138,11 +15140,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15141: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15143: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:15145: \$? = $ac_status" >&5 + echo "$as_me:15147: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -15242,11 +15244,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15245: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15247: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:15249: \$? = $ac_status" >&5 + echo "$as_me:15251: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -17694,7 +17696,7 @@ lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext < conftest.$ac_ext <&5) + (eval echo "\"\$as_me:20167: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:20169: \$? = $ac_status" >&5 + echo "$as_me:20171: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -20266,11 +20268,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:20269: $lt_compile\"" >&5) + (eval echo "\"\$as_me:20271: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:20273: \$? = $ac_status" >&5 + echo "$as_me:20275: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -21836,11 +21838,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:21839: $lt_compile\"" >&5) + (eval echo "\"\$as_me:21841: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:21843: \$? = $ac_status" >&5 + echo "$as_me:21845: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -21940,11 +21942,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:21943: $lt_compile\"" >&5) + (eval echo "\"\$as_me:21945: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:21947: \$? = $ac_status" >&5 + echo "$as_me:21949: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -24175,11 +24177,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:24178: $lt_compile\"" >&5) + (eval echo "\"\$as_me:24180: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:24182: \$? = $ac_status" >&5 + echo "$as_me:24184: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -24443,11 +24445,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:24446: $lt_compile\"" >&5) + (eval echo "\"\$as_me:24448: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:24450: \$? = $ac_status" >&5 + echo "$as_me:24452: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. @@ -24547,11 +24549,11 @@ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:24550: $lt_compile\"" >&5) + (eval echo "\"\$as_me:24552: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:24554: \$? = $ac_status" >&5 + echo "$as_me:24556: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -27459,6 +27461,15 @@ { echo "$as_me:$LINENO: result: ok" >&5 echo "${ECHO_T}ok" >&6; } +{ echo "$as_me:$LINENO: checking optional compiler flags" >&5 +echo $ECHO_N "checking optional compiler flags... $ECHO_C" >&6; } +NO_VARIADIC_MACROS=`$CXX -Wno-variadic-macros -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-variadic-macros` + +NO_MISSING_FIELD_INITIALIZERS=`$CXX -Wno-missing-field-initializers -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-missing-field-initializers` + +{ echo "$as_me:$LINENO: result: $NO_VARIADIC_MACROS $NO_MISSING_FIELD_INITIALIZERS" >&5 +echo "${ECHO_T}$NO_VARIADIC_MACROS $NO_MISSING_FIELD_INITIALIZERS" >&6; } + { echo "$as_me:$LINENO: checking for sin in -lm" >&5 @@ -36167,6 +36178,8 @@ LLVMGXXCOMMAND!$LLVMGXXCOMMAND$ac_delim LLVMGCC!$LLVMGCC$ac_delim LLVMGXX!$LLVMGXX$ac_delim +NO_VARIADIC_MACROS!$NO_VARIADIC_MACROS$ac_delim +NO_MISSING_FIELD_INITIALIZERS!$NO_MISSING_FIELD_INITIALIZERS$ac_delim USE_UDIS86!$USE_UDIS86$ac_delim USE_OPROFILE!$USE_OPROFILE$ac_delim HAVE_PTHREAD!$HAVE_PTHREAD$ac_delim @@ -36201,7 +36214,7 @@ LTLIBOBJS!$LTLIBOBJS$ac_delim _ACEOF - if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 92; then + if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 94; then break elif $ac_last_try; then { { echo "$as_me:$LINENO: error: could not make $CONFIG_STATUS" >&5 From resistor at me.com Mon Oct 26 14:58:09 2009 From: resistor at me.com (Owen Anderson) Date: Mon, 26 Oct 2009 12:58:09 -0700 Subject: [llvm-commits] [llvm] r85144 - in /llvm/trunk: include/llvm/LinkAllPasses.h include/llvm/Transforms/Scalar.h lib/Transforms/Scalar/GEPSplitter.cpp In-Reply-To: <200910261912.n9QJCE4a030428@zion.cs.uiuc.edu> References: <200910261912.n9QJCE4a030428@zion.cs.uiuc.edu> Message-ID: <83489F21-6E25-42FB-B136-84DDBE36BF0E@me.com> On Oct 26, 2009, at 12:12 PM, Dan Gohman wrote: > Author: djg > Date: Mon Oct 26 14:12:14 2009 > New Revision: 85144 > > URL: http://llvm.org/viewvc/llvm-project?rev=85144&view=rev > Log: > Check in the experimental GEP splitter pass. This pass splits complex > GEPs (more than one non-zero index) into simple GEPs (at most one > non-zero index). In some simple experiments using this it's not > uncommon to see 3% overall code size wins, because it exposes > redundancies that can be eliminated, however it's tricky to use > because instcombine aggressively undoes the work that this pass does. > How about running it right before GVN, and letting a later instcombine pass clean it up? --Owen > Added: > llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp > Modified: > llvm/trunk/include/llvm/LinkAllPasses.h > llvm/trunk/include/llvm/Transforms/Scalar.h > > Modified: llvm/trunk/include/llvm/LinkAllPasses.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LinkAllPasses.h?rev=85144&r1=85143&r2=85144&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/include/llvm/LinkAllPasses.h (original) > +++ llvm/trunk/include/llvm/LinkAllPasses.h Mon Oct 26 14:12:14 2009 > @@ -141,6 +141,7 @@ > (void) llvm::createPartialInliningPass(); > (void) llvm::createSSIPass(); > (void) llvm::createSSIEverythingPass(); > + (void) llvm::createGEPSplitterPass(); > > (void)new llvm::IntervalPartition(); > (void)new llvm::FindUsedTypes(); > > Modified: llvm/trunk/include/llvm/Transforms/Scalar.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Scalar.h?rev=85144&r1=85143&r2=85144&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/include/llvm/Transforms/Scalar.h (original) > +++ llvm/trunk/include/llvm/Transforms/Scalar.h Mon Oct 26 14:12:14 > 2009 > @@ -341,6 +341,12 @@ > // > FunctionPass *createSSIEverythingPass(); > > +// > = > = > = > ----------------------------------------------------------------------= > ==// > +// > +// GEPSplitter - Split complex GEPs into simple ones > +// > +FunctionPass *createGEPSplitterPass(); > + > } // End llvm namespace > > #endif > > Added: llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp?rev=85144&view=auto > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp (added) > +++ llvm/trunk/lib/Transforms/Scalar/GEPSplitter.cpp Mon Oct 26 > 14:12:14 2009 > @@ -0,0 +1,81 @@ > +//===- GEPSplitter.cpp - Split complex GEPs into simple ones > --------------===// > +// > +// The LLVM Compiler Infrastructure > +// > +// This file is distributed under the University of Illinois Open > Source > +// License. See LICENSE.TXT for details. > +// > +// > = > = > = > ----------------------------------------------------------------------= > ==// > +// > +// This function breaks GEPs with more than 2 non-zero operands > into smaller > +// GEPs each with no more than 2 non-zero operands. This exposes > redundancy > +// between GEPs with common initial operand sequences. > +// > +// > = > = > = > ----------------------------------------------------------------------= > ==// > + > +#define DEBUG_TYPE "split-geps" > +#include "llvm/Transforms/Scalar.h" > +#include "llvm/Constants.h" > +#include "llvm/Function.h" > +#include "llvm/Instructions.h" > +#include "llvm/Pass.h" > +using namespace llvm; > + > +namespace { > + class GEPSplitter : public FunctionPass { > + virtual bool runOnFunction(Function &F); > + virtual void getAnalysisUsage(AnalysisUsage &AU) const; > + public: > + static char ID; // Pass identification, replacement for typeid > + explicit GEPSplitter() : FunctionPass(&ID) {} > + }; > +} > + > +char GEPSplitter::ID = 0; > +static RegisterPass X("split-geps", > + "split complex GEPs into simple > GEPs"); > + > +FunctionPass *llvm::createGEPSplitterPass() { > + return new GEPSplitter(); > +} > + > +bool GEPSplitter::runOnFunction(Function &F) { > + bool Changed = false; > + > + // Visit each GEP instruction. > + for (Function::iterator I = F.begin(), E = F.end(); I != E; ++I) > + for (BasicBlock::iterator II = I->begin(), IE = I->end(); II != > IE; ) > + if (GetElementPtrInst *GEP = dyn_cast(II+ > +)) { > + unsigned NumOps = GEP->getNumOperands(); > + // Ignore GEPs which are already simple. > + if (NumOps <= 2) > + continue; > + bool FirstIndexIsZero = isa(GEP->getOperand > (1)) && > + cast(GEP->getOperand > (1))->isZero(); > + if (NumOps == 3 && FirstIndexIsZero) > + continue; > + // The first index is special and gets expanded with a 2- > operand GEP > + // (unless it's zero, in which case we can skip this). > + Value *NewGEP = FirstIndexIsZero ? > + GEP->getOperand(0) : > + GetElementPtrInst::Create(GEP->getOperand(0), GEP- > >getOperand(1), > + "tmp", GEP); > + // All remaining indices get expanded with a 3-operand GEP > with zero > + // as the second operand. > + Value *Idxs[2]; > + Idxs[0] = ConstantInt::get(Type::getInt64Ty(F.getContext > ()), 0); > + for (unsigned i = 2; i != NumOps; ++i) { > + Idxs[1] = GEP->getOperand(i); > + NewGEP = GetElementPtrInst::Create(NewGEP, Idxs, Idxs+2, > "tmp", GEP); > + } > + GEP->replaceAllUsesWith(NewGEP); > + GEP->eraseFromParent(); > + Changed = true; > + } > + > + return Changed; > +} > + > +void GEPSplitter::getAnalysisUsage(AnalysisUsage &AU) const { > + AU.setPreservesCFG(); > +} > > > _______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits From jlerouge at apple.com Mon Oct 26 15:01:35 2009 From: jlerouge at apple.com (Julien Lerouge) Date: Mon, 26 Oct 2009 20:01:35 -0000 Subject: [llvm-commits] [llvm] r85149 - in /llvm/trunk: unittests/Makefile.unittest utils/unittest/UnitTestMain/Makefile utils/unittest/googletest/Makefile Message-ID: <200910262001.n9QK1ajv000600@zion.cs.uiuc.edu> Author: jlerouge Date: Mon Oct 26 15:01:35 2009 New Revision: 85149 URL: http://llvm.org/viewvc/llvm-project?rev=85149&view=rev Log: Remove / use flags that are now set in the Makefile.config. Modified: llvm/trunk/unittests/Makefile.unittest llvm/trunk/utils/unittest/UnitTestMain/Makefile llvm/trunk/utils/unittest/googletest/Makefile Modified: llvm/trunk/unittests/Makefile.unittest URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Makefile.unittest?rev=85149&r1=85148&r2=85149&view=diff ============================================================================== --- llvm/trunk/unittests/Makefile.unittest (original) +++ llvm/trunk/unittests/Makefile.unittest Mon Oct 26 15:01:35 2009 @@ -19,7 +19,7 @@ LLVMUnitTestExe = $(BuildMode)/$(TESTNAME)Tests$(EXEEXT) CPP.Flags += -I$(LLVM_SRC_ROOT)/utils/unittest/googletest/include/ -CPP.Flags += -Wno-variadic-macros +CPP.Flags += $(NO_VARIADIC_MACROS) TESTLIBS = -lGoogleTest -lUnitTestMain $(LLVMUnitTestExe): $(ObjectsO) $(ProjLibsPaths) $(LLVMLibsPaths) Modified: llvm/trunk/utils/unittest/UnitTestMain/Makefile URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/unittest/UnitTestMain/Makefile?rev=85149&r1=85148&r2=85149&view=diff ============================================================================== --- llvm/trunk/utils/unittest/UnitTestMain/Makefile (original) +++ llvm/trunk/utils/unittest/UnitTestMain/Makefile Mon Oct 26 15:01:35 2009 @@ -10,8 +10,6 @@ LEVEL = ../../.. include $(LEVEL)/Makefile.config -NO_MISSING_FIELD_INITIALIZERS := $(shell $(CXX) -Wno-missing-field-initializers -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-missing-field-initializers) -NO_VARIADIC_MACROS := $(shell $(CXX) -Wno-variadic-macros -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-variadic-macros) LIBRARYNAME = UnitTestMain BUILD_ARCHIVE = 1 Modified: llvm/trunk/utils/unittest/googletest/Makefile URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/unittest/googletest/Makefile?rev=85149&r1=85148&r2=85149&view=diff ============================================================================== --- llvm/trunk/utils/unittest/googletest/Makefile (original) +++ llvm/trunk/utils/unittest/googletest/Makefile Mon Oct 26 15:01:35 2009 @@ -10,8 +10,6 @@ LEVEL := ../../.. include $(LEVEL)/Makefile.config -NO_MISSING_FIELD_INITIALIZERS := $(shell $(CXX) -Wno-missing-field-initializers -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-missing-field-initializers) -NO_VARIADIC_MACROS := $(shell $(CXX) -Wno-variadic-macros -fsyntax-only -xc /dev/null 2>/dev/null && echo -Wno-variadic-macros) LIBRARYNAME = GoogleTest BUILD_ARCHIVE = 1 From gohman at apple.com Mon Oct 26 15:34:45 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 13:34:45 -0700 Subject: [llvm-commits] [llvm] r85144 - in /llvm/trunk: include/llvm/LinkAllPasses.h include/llvm/Transforms/Scalar.h lib/Transforms/Scalar/GEPSplitter.cpp In-Reply-To: <83489F21-6E25-42FB-B136-84DDBE36BF0E@me.com> References: <200910261912.n9QJCE4a030428@zion.cs.uiuc.edu> <83489F21-6E25-42FB-B136-84DDBE36BF0E@me.com> Message-ID: <9C3EC32B-AE0D-462E-A6FD-2307ED306406@apple.com> On Oct 26, 2009, at 12:58 PM, Owen Anderson wrote: > > On Oct 26, 2009, at 12:12 PM, Dan Gohman wrote: > >> Author: djg >> Date: Mon Oct 26 14:12:14 2009 >> New Revision: 85144 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=85144&view=rev >> Log: >> Check in the experimental GEP splitter pass. This pass splits complex >> GEPs (more than one non-zero index) into simple GEPs (at most one >> non-zero index). In some simple experiments using this it's not >> uncommon to see 3% overall code size wins, because it exposes >> redundancies that can be eliminated, however it's tricky to use >> because instcombine aggressively undoes the work that this pass does. >> > > > How about running it right before GVN, and letting a later instcombine > pass clean it up? Instcombine currently merges GEPs even when they have multiple uses, so it defeats even this. And it's not entirely unjustified: BasicAA understands standalone complex GEPs better than chains of simple GEPs. Dan From wendling at apple.com Mon Oct 26 15:37:33 2009 From: wendling at apple.com (Bill Wendling) Date: Mon, 26 Oct 2009 13:37:33 -0700 Subject: [llvm-commits] [llvm] r85141 - /llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h In-Reply-To: References: <200910261900.n9QJ0lP6029737@zion.cs.uiuc.edu>

Message-ID: <9E8C82EF-C4C2-46BF-9F55-DE25AB0D6B84@apple.com> Thanks! :-) -bw On Oct 26, 2009, at 12:41 PM, David Goodwin wrote: > Sending include/llvm/CodeGen/AntiDepBreaker.h > Sending lib/CodeGen/PostRASchedulerList.cpp > Transmitting file data .. > Committed revision 85146. > > On Oct 26, 2009, at 12:07 PM, Bill Wendling wrote: > >> On Oct 26, 2009, at 12:00 PM, David Goodwin wrote: >> >>> Author: david_goodwin >>> Date: Mon Oct 26 14:00:47 2009 >>> New Revision: 85141 >>> >>> URL: http://llvm.org/viewvc/llvm-project?rev=85141&view=rev >>> Log: >>> Add virtual destructor. >>> >>> Modified: >>> llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >>> >>> Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85141&r1=85140&r2=85141&view=diff >>> >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> ==================================================================== >>> --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) >>> +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 >>> 14:00:47 2009 >>> @@ -29,6 +29,8 @@ >>> /// anti-dependencies. >>> class AntiDepBreaker { >>> public: >>> + virtual ~AntiDepBreaker() { }; >>> + >> >> It's good to have the definition of a virtual destructor in >> the .cpp file. It "anchors" the vtable. >> >> -bw >> > From evan.cheng at apple.com Mon Oct 26 16:23:32 2009 From: evan.cheng at apple.com (Evan Cheng) Date: Mon, 26 Oct 2009 14:23:32 -0700 Subject: [llvm-commits] [llvm] r85127 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/CMakeLists.txt lib/CodeGen/CriticalAntiDepBreaker.cpp lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp lib/Target/ARM/ARMSubtarget.h test/CodeGen/X86/break-anti-dependencies.ll In-Reply-To: <200910261659.n9QGx4Yj023936@zion.cs.uiuc.edu> References: <200910261659.n9QGx4Yj023936@zion.cs.uiuc.edu> Message-ID: It seems to be AntiDepBreaker.h should be inside lib/CodeGen. Is there a reason to expose it? Evan On Oct 26, 2009, at 9:59 AM, David Goodwin wrote: > Author: david_goodwin > Date: Mon Oct 26 11:59:04 2009 > New Revision: 85127 > > URL: http://llvm.org/viewvc/llvm-project?rev=85127&view=rev > Log: > Break anti-dependence breaking out into its own class. > > Added: > llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h > llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp > llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h > Modified: > llvm/trunk/lib/CodeGen/CMakeLists.txt > llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp > llvm/trunk/lib/Target/ARM/ARMSubtarget.h > llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll > > Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85127&view=auto > > ============================================================================== > --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (added) > +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 11:59:04 2009 > @@ -0,0 +1,56 @@ > +//=- llvm/CodeGen/AntiDepBreaker.h - Anti-Dependence Breaking -*- C++ -*-=// > +// > +// The LLVM Compiler Infrastructure > +// > +// This file is distributed under the University of Illinois Open Source > +// License. See LICENSE.TXT for details. > +// > +//===----------------------------------------------------------------------===// > +// > +// This file implements the AntiDepBreaker class, which implements > +// anti-dependence breaking heuristics for post-register-allocation scheduling. > +// > +//===----------------------------------------------------------------------===// > + > +#ifndef LLVM_CODEGEN_ANTIDEPBREAKER_H > +#define LLVM_CODEGEN_ANTIDEPBREAKER_H > + > +#include "llvm/CodeGen/MachineBasicBlock.h" > +#include "llvm/CodeGen/MachineFrameInfo.h" > +#include "llvm/CodeGen/MachineFunction.h" > +#include "llvm/CodeGen/MachineRegisterInfo.h" > +#include "llvm/CodeGen/ScheduleDAG.h" > +#include "llvm/Target/TargetRegisterInfo.h" > + > +namespace llvm { > + > +/// AntiDepBreaker - This class works into conjunction with the > +/// post-RA scheduler to rename registers to break register > +/// anti-dependencies. > +class AntiDepBreaker { > +public: > + /// Start - Initialize anti-dep breaking for a new basic block. > + virtual void StartBlock(MachineBasicBlock *BB) =0; > + > + /// BreakAntiDependencies - Identifiy anti-dependencies within a > + /// basic-block region and break them by renaming registers. Return > + /// the number of anti-dependencies broken. > + /// > + virtual unsigned BreakAntiDependencies(std::vector& SUnits, > + MachineBasicBlock::iterator& Begin, > + MachineBasicBlock::iterator& End, > + unsigned InsertPosIndex) =0; > + > + /// Observe - Update liveness information to account for the current > + /// instruction, which will not be scheduled. > + /// > + virtual void Observe(MachineInstr *MI, unsigned Count, > + unsigned InsertPosIndex) =0; > + > + /// Finish - Finish anti-dep breaking for a basic block. > + virtual void FinishBlock() =0; > +}; > + > +} > + > +#endif > > Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85127&r1=85126&r2=85127&view=diff > > ============================================================================== > --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) > +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 11:59:04 2009 > @@ -1,6 +1,7 @@ > add_llvm_library(LLVMCodeGen > BranchFolding.cpp > CodePlacementOpt.cpp > + CriticalAntiDepBreaker.cpp > DeadMachineInstructionElim.cpp > DwarfEHPrepare.cpp > ELFCodeEmitter.cpp > > Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp?rev=85127&view=auto > > ============================================================================== > --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp (added) > +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp Mon Oct 26 11:59:04 2009 > @@ -0,0 +1,539 @@ > +//===----- CriticalAntiDepBreaker.cpp - Anti-dep breaker -------- ---------===// > +// > +// The LLVM Compiler Infrastructure > +// > +// This file is distributed under the University of Illinois Open Source > +// License. See LICENSE.TXT for details. > +// > +//===----------------------------------------------------------------------===// > +// > +// This file implements the CriticalAntiDepBreaker class, which > +// implements register anti-dependence breaking along a blocks > +// critical path during post-RA scheduler. > +// > +//===----------------------------------------------------------------------===// > + > +#define DEBUG_TYPE "critical-antidep" > +#include "CriticalAntiDepBreaker.h" > +#include "llvm/CodeGen/MachineBasicBlock.h" > +#include "llvm/CodeGen/MachineFrameInfo.h" > +#include "llvm/Target/TargetMachine.h" > +#include "llvm/Target/TargetRegisterInfo.h" > +#include "llvm/Support/Debug.h" > +#include "llvm/Support/ErrorHandling.h" > +#include "llvm/Support/raw_ostream.h" > + > +using namespace llvm; > + > +CriticalAntiDepBreaker:: > +CriticalAntiDepBreaker(MachineFunction& MFi) : > + AntiDepBreaker(), MF(MFi), > + MRI(MF.getRegInfo()), > + TRI(MF.getTarget().getRegisterInfo()), > + AllocatableSet(TRI->getAllocatableSet(MF)) > +{ > +} > + > +CriticalAntiDepBreaker::~CriticalAntiDepBreaker() { > +} > + > +void CriticalAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { > + // Clear out the register class data. > + std::fill(Classes, array_endof(Classes), > + static_cast(0)); > + > + // Initialize the indices to indicate that no registers are live. > + std::fill(KillIndices, array_endof(KillIndices), ~0u); > + std::fill(DefIndices, array_endof(DefIndices), BB->size()); > + > + // Clear "do not change" set. > + KeepRegs.clear(); > + > + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); > + > + // Determine the live-out physregs for this block. > + if (IsReturnBlock) { > + // In a return block, examine the function live-out regs. > + for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), > + E = MRI.liveout_end(); I != E; ++I) { > + unsigned Reg = *I; > + Classes[Reg] = reinterpret_cast(-1); > + KillIndices[Reg] = BB->size(); > + DefIndices[Reg] = ~0u; > + // Repeat, for all aliases. > + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > + unsigned AliasReg = *Alias; > + Classes[AliasReg] = reinterpret_cast(-1); > + KillIndices[AliasReg] = BB->size(); > + DefIndices[AliasReg] = ~0u; > + } > + } > + } else { > + // In a non-return block, examine the live-in regs of all successors. > + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), > + SE = BB->succ_end(); SI != SE; ++SI) > + for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), > + E = (*SI)->livein_end(); I != E; ++I) { > + unsigned Reg = *I; > + Classes[Reg] = reinterpret_cast(-1); > + KillIndices[Reg] = BB->size(); > + DefIndices[Reg] = ~0u; > + // Repeat, for all aliases. > + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > + unsigned AliasReg = *Alias; > + Classes[AliasReg] = reinterpret_cast(-1); > + KillIndices[AliasReg] = BB->size(); > + DefIndices[AliasReg] = ~0u; > + } > + } > + } > + > + // Mark live-out callee-saved registers. In a return block this is > + // all callee-saved registers. In non-return this is any > + // callee-saved register that is not saved in the prolog. > + const MachineFrameInfo *MFI = MF.getFrameInfo(); > + BitVector Pristine = MFI->getPristineRegs(BB); > + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { > + unsigned Reg = *I; > + if (!IsReturnBlock && !Pristine.test(Reg)) continue; > + Classes[Reg] = reinterpret_cast(-1); > + KillIndices[Reg] = BB->size(); > + DefIndices[Reg] = ~0u; > + // Repeat, for all aliases. > + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > + unsigned AliasReg = *Alias; > + Classes[AliasReg] = reinterpret_cast(-1); > + KillIndices[AliasReg] = BB->size(); > + DefIndices[AliasReg] = ~0u; > + } > + } > +} > + > +void CriticalAntiDepBreaker::FinishBlock() { > + RegRefs.clear(); > + KeepRegs.clear(); > +} > + > +void CriticalAntiDepBreaker::Observe(MachineInstr *MI, unsigned Count, > + unsigned InsertPosIndex) { > + assert(Count < InsertPosIndex && "Instruction index out of expected range!"); > + > + // Any register which was defined within the previous scheduling region > + // may have been rescheduled and its lifetime may overlap with registers > + // in ways not reflected in our current liveness state. For each such > + // register, adjust the liveness state to be conservatively correct. > + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) > + if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { > + assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); > + // Mark this register to be non-renamable. > + Classes[Reg] = reinterpret_cast(-1); > + // Move the def index to the end of the previous region, to reflect > + // that the def could theoretically have been scheduled at the end. > + DefIndices[Reg] = InsertPosIndex; > + } > + > + PrescanInstruction(MI); > + ScanInstruction(MI, Count); > +} > + > +/// CriticalPathStep - Return the next SUnit after SU on the bottom-up > +/// critical path. > +static SDep *CriticalPathStep(SUnit *SU) { > + SDep *Next = 0; > + unsigned NextDepth = 0; > + // Find the predecessor edge with the greatest depth. > + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); > + P != PE; ++P) { > + SUnit *PredSU = P->getSUnit(); > + unsigned PredLatency = P->getLatency(); > + unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; > + // In the case of a latency tie, prefer an anti-dependency edge over > + // other types of edges. > + if (NextDepth < PredTotalLatency || > + (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { > + NextDepth = PredTotalLatency; > + Next = &*P; > + } > + } > + return Next; > +} > + > +void CriticalAntiDepBreaker::PrescanInstruction(MachineInstr *MI) { > + // Scan the register operands for this instruction and update > + // Classes and RegRefs. > + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > + MachineOperand &MO = MI->getOperand(i); > + if (!MO.isReg()) continue; > + unsigned Reg = MO.getReg(); > + if (Reg == 0) continue; > + const TargetRegisterClass *NewRC = 0; > + > + if (i < MI->getDesc().getNumOperands()) > + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); > + > + // For now, only allow the register to be changed if its register > + // class is consistent across all uses. > + if (!Classes[Reg] && NewRC) > + Classes[Reg] = NewRC; > + else if (!NewRC || Classes[Reg] != NewRC) > + Classes[Reg] = reinterpret_cast(-1); > + > + // Now check for aliases. > + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > + // If an alias of the reg is used during the live range, give up. > + // Note that this allows us to skip checking if AntiDepReg > + // overlaps with any of the aliases, among other things. > + unsigned AliasReg = *Alias; > + if (Classes[AliasReg]) { > + Classes[AliasReg] = reinterpret_cast(-1); > + Classes[Reg] = reinterpret_cast(-1); > + } > + } > + > + // If we're still willing to consider this register, note the reference. > + if (Classes[Reg] != reinterpret_cast(-1)) > + RegRefs.insert(std::make_pair(Reg, &MO)); > + > + // It's not safe to change register allocation for source operands of > + // that have special allocation requirements. > + if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { > + if (KeepRegs.insert(Reg)) { > + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); > + *Subreg; ++Subreg) > + KeepRegs.insert(*Subreg); > + } > + } > + } > +} > + > +void CriticalAntiDepBreaker::ScanInstruction(MachineInstr *MI, > + unsigned Count) { > + // Update liveness. > + // Proceding upwards, registers that are defed but not used in this > + // instruction are now dead. > + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > + MachineOperand &MO = MI->getOperand(i); > + if (!MO.isReg()) continue; > + unsigned Reg = MO.getReg(); > + if (Reg == 0) continue; > + if (!MO.isDef()) continue; > + // Ignore two-addr defs. > + if (MI->isRegTiedToUseOperand(i)) continue; > + > + DefIndices[Reg] = Count; > + KillIndices[Reg] = ~0u; > + assert(((KillIndices[Reg] == ~0u) != > + (DefIndices[Reg] == ~0u)) && > + "Kill and Def maps aren't consistent for Reg!"); > + KeepRegs.erase(Reg); > + Classes[Reg] = 0; > + RegRefs.erase(Reg); > + // Repeat, for all subregs. > + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); > + *Subreg; ++Subreg) { > + unsigned SubregReg = *Subreg; > + DefIndices[SubregReg] = Count; > + KillIndices[SubregReg] = ~0u; > + KeepRegs.erase(SubregReg); > + Classes[SubregReg] = 0; > + RegRefs.erase(SubregReg); > + } > + // Conservatively mark super-registers as unusable. > + for (const unsigned *Super = TRI->getSuperRegisters(Reg); > + *Super; ++Super) { > + unsigned SuperReg = *Super; > + Classes[SuperReg] = reinterpret_cast(-1); > + } > + } > + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > + MachineOperand &MO = MI->getOperand(i); > + if (!MO.isReg()) continue; > + unsigned Reg = MO.getReg(); > + if (Reg == 0) continue; > + if (!MO.isUse()) continue; > + > + const TargetRegisterClass *NewRC = 0; > + if (i < MI->getDesc().getNumOperands()) > + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); > + > + // For now, only allow the register to be changed if its register > + // class is consistent across all uses. > + if (!Classes[Reg] && NewRC) > + Classes[Reg] = NewRC; > + else if (!NewRC || Classes[Reg] != NewRC) > + Classes[Reg] = reinterpret_cast(-1); > + > + RegRefs.insert(std::make_pair(Reg, &MO)); > + > + // It wasn't previously live but now it is, this is a kill. > + if (KillIndices[Reg] == ~0u) { > + KillIndices[Reg] = Count; > + DefIndices[Reg] = ~0u; > + assert(((KillIndices[Reg] == ~0u) != > + (DefIndices[Reg] == ~0u)) && > + "Kill and Def maps aren't consistent for Reg!"); > + } > + // Repeat, for all aliases. > + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > + unsigned AliasReg = *Alias; > + if (KillIndices[AliasReg] == ~0u) { > + KillIndices[AliasReg] = Count; > + DefIndices[AliasReg] = ~0u; > + } > + } > + } > +} > + > +unsigned > +CriticalAntiDepBreaker::findSuitableFreeRegister(unsigned AntiDepReg, > + unsigned LastNewReg, > + const TargetRegisterClass *RC) { > + for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), > + RE = RC->allocation_order_end(MF); R != RE; ++R) { > + unsigned NewReg = *R; > + // Don't replace a register with itself. > + if (NewReg == AntiDepReg) continue; > + // Don't replace a register with one that was recently used to repair > + // an anti-dependence with this AntiDepReg, because that would > + // re-introduce that anti-dependence. > + if (NewReg == LastNewReg) continue; > + // If NewReg is dead and NewReg's most recent def is not before > + // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. > + assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && > + "Kill and Def maps aren't consistent for AntiDepReg!"); > + assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && > + "Kill and Def maps aren't consistent for NewReg!"); > + if (KillIndices[NewReg] != ~0u || > + Classes[NewReg] == reinterpret_cast(-1) || > + KillIndices[AntiDepReg] > DefIndices[NewReg]) > + continue; > + return NewReg; > + } > + > + // No registers are free and available! > + return 0; > +} > + > +unsigned CriticalAntiDepBreaker:: > +BreakAntiDependencies(std::vector& SUnits, > + MachineBasicBlock::iterator& Begin, > + MachineBasicBlock::iterator& End, > + unsigned InsertPosIndex) { > + // The code below assumes that there is at least one instruction, > + // so just duck out immediately if the block is empty. > + if (SUnits.empty()) return 0; > + > + // Find the node at the bottom of the critical path. > + SUnit *Max = 0; > + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { > + SUnit *SU = &SUnits[i]; > + if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) > + Max = SU; > + } > + > +#ifndef NDEBUG > + { > + DEBUG(errs() << "Critical path has total latency " > + << (Max->getDepth() + Max->Latency) << "\n"); > + DEBUG(errs() << "Available regs:"); > + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { > + if (KillIndices[Reg] == ~0u) > + DEBUG(errs() << " " << TRI->getName(Reg)); > + } > + DEBUG(errs() << '\n'); > + } > +#endif > + > + // Track progress along the critical path through the SUnit graph as we walk > + // the instructions. > + SUnit *CriticalPathSU = Max; > + MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); > + > + // Consider this pattern: > + // A = ... > + // ... = A > + // A = ... > + // ... = A > + // A = ... > + // ... = A > + // A = ... > + // ... = A > + // There are three anti-dependencies here, and without special care, > + // we'd break all of them using the same register: > + // A = ... > + // ... = A > + // B = ... > + // ... = B > + // B = ... > + // ... = B > + // B = ... > + // ... = B > + // because at each anti-dependence, B is the first register that > + // isn't A which is free. This re-introduces anti-dependencies > + // at all but one of the original anti-dependencies that we were > + // trying to break. To avoid this, keep track of the most recent > + // register that each register was replaced with, avoid > + // using it to repair an anti-dependence on the same register. > + // This lets us produce this: > + // A = ... > + // ... = A > + // B = ... > + // ... = B > + // C = ... > + // ... = C > + // B = ... > + // ... = B > + // This still has an anti-dependence on B, but at least it isn't on the > + // original critical path. > + // > + // TODO: If we tracked more than one register here, we could potentially > + // fix that remaining critical edge too. This is a little more involved, > + // because unlike the most recent register, less recent registers should > + // still be considered, though only if no other registers are available. > + unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; > + > + // Attempt to break anti-dependence edges on the critical path. Walk the > + // instructions from the bottom up, tracking information about liveness > + // as we go to help determine which registers are available. > + unsigned Broken = 0; > + unsigned Count = InsertPosIndex - 1; > + for (MachineBasicBlock::iterator I = End, E = Begin; > + I != E; --Count) { > + MachineInstr *MI = --I; > + > + // Check if this instruction has a dependence on the critical path that > + // is an anti-dependence that we may be able to break. If it is, set > + // AntiDepReg to the non-zero register associated with the anti-dependence. > + // > + // We limit our attention to the critical path as a heuristic to avoid > + // breaking anti-dependence edges that aren't going to significantly > + // impact the overall schedule. There are a limited number of registers > + // and we want to save them for the important edges. > + // > + // TODO: Instructions with multiple defs could have multiple > + // anti-dependencies. The current code here only knows how to break one > + // edge per instruction. Note that we'd have to be able to break all of > + // the anti-dependencies in an instruction in order to be effective. > + unsigned AntiDepReg = 0; > + if (MI == CriticalPathMI) { > + if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { > + SUnit *NextSU = Edge->getSUnit(); > + > + // Only consider anti-dependence edges. > + if (Edge->getKind() == SDep::Anti) { > + AntiDepReg = Edge->getReg(); > + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); > + if (!AllocatableSet.test(AntiDepReg)) > + // Don't break anti-dependencies on non-allocatable registers. > + AntiDepReg = 0; > + else if (KeepRegs.count(AntiDepReg)) > + // Don't break anti-dependencies if an use down below requires > + // this exact register. > + AntiDepReg = 0; > + else { > + // If the SUnit has other dependencies on the SUnit that it > + // anti-depends on, don't bother breaking the anti-dependency > + // since those edges would prevent such units from being > + // scheduled past each other regardless. > + // > + // Also, if there are dependencies on other SUnits with the > + // same register as the anti-dependency, don't attempt to > + // break it. > + for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), > + PE = CriticalPathSU->Preds.end(); P != PE; ++P) > + if (P->getSUnit() == NextSU ? > + (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : > + (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { > + AntiDepReg = 0; > + break; > + } > + } > + } > + CriticalPathSU = NextSU; > + CriticalPathMI = CriticalPathSU->getInstr(); > + } else { > + // We've reached the end of the critical path. > + CriticalPathSU = 0; > + CriticalPathMI = 0; > + } > + } > + > + PrescanInstruction(MI); > + > + if (MI->getDesc().hasExtraDefRegAllocReq()) > + // If this instruction's defs have special allocation requirement, don't > + // break this anti-dependency. > + AntiDepReg = 0; > + else if (AntiDepReg) { > + // If this instruction has a use of AntiDepReg, breaking it > + // is invalid. > + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > + MachineOperand &MO = MI->getOperand(i); > + if (!MO.isReg()) continue; > + unsigned Reg = MO.getReg(); > + if (Reg == 0) continue; > + if (MO.isUse() && AntiDepReg == Reg) { > + AntiDepReg = 0; > + break; > + } > + } > + } > + > + // Determine AntiDepReg's register class, if it is live and is > + // consistently used within a single class. > + const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; > + assert((AntiDepReg == 0 || RC != NULL) && > + "Register should be live if it's causing an anti-dependence!"); > + if (RC == reinterpret_cast(-1)) > + AntiDepReg = 0; > + > + // Look for a suitable register to use to break the anti-depenence. > + // > + // TODO: Instead of picking the first free register, consider which might > + // be the best. > + if (AntiDepReg != 0) { > + if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, > + LastNewReg[AntiDepReg], > + RC)) { > + DEBUG(errs() << "Breaking anti-dependence edge on " > + << TRI->getName(AntiDepReg) > + << " with " << RegRefs.count(AntiDepReg) << " references" > + << " using " << TRI->getName(NewReg) << "!\n"); > + > + // Update the references to the old register to refer to the new > + // register. > + std::pair::iterator, > + std::multimap::iterator> > + Range = RegRefs.equal_range(AntiDepReg); > + for (std::multimap::iterator > + Q = Range.first, QE = Range.second; Q != QE; ++Q) > + Q->second->setReg(NewReg); > + > + // We just went back in time and modified history; the > + // liveness information for the anti-depenence reg is now > + // inconsistent. Set the state as if it were dead. > + Classes[NewReg] = Classes[AntiDepReg]; > + DefIndices[NewReg] = DefIndices[AntiDepReg]; > + KillIndices[NewReg] = KillIndices[AntiDepReg]; > + assert(((KillIndices[NewReg] == ~0u) != > + (DefIndices[NewReg] == ~0u)) && > + "Kill and Def maps aren't consistent for NewReg!"); > + > + Classes[AntiDepReg] = 0; > + DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; > + KillIndices[AntiDepReg] = ~0u; > + assert(((KillIndices[AntiDepReg] == ~0u) != > + (DefIndices[AntiDepReg] == ~0u)) && > + "Kill and Def maps aren't consistent for AntiDepReg!"); > + > + RegRefs.erase(AntiDepReg); > + LastNewReg[AntiDepReg] = NewReg; > + ++Broken; > + } > + } > + > + ScanInstruction(MI, Count); > + } > + > + return Broken; > +} > > Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85127&view=auto > > ============================================================================== > --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (added) > +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 11:59:04 2009 > @@ -0,0 +1,95 @@ > +//=- llvm/CodeGen/CriticalAntiDepBreaker.h - Anti-Dep Support -*- C++ -*-=// > +// > +// The LLVM Compiler Infrastructure > +// > +// This file is distributed under the University of Illinois Open Source > +// License. See LICENSE.TXT for details. > +// > +//===----------------------------------------------------------------------===// > +// > +// This file implements the CriticalAntiDepBreaker class, which > +// implements register anti-dependence breaking along a blocks > +// critical path during post-RA scheduler. > +// > +//===----------------------------------------------------------------------===// > + > +#ifndef LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H > +#define LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H > + > +#include "llvm/CodeGen/AntiDepBreaker.h" > +#include "llvm/CodeGen/MachineBasicBlock.h" > +#include "llvm/CodeGen/MachineFrameInfo.h" > +#include "llvm/CodeGen/MachineFunction.h" > +#include "llvm/CodeGen/MachineRegisterInfo.h" > +#include "llvm/CodeGen/ScheduleDAG.h" > +#include "llvm/Target/TargetRegisterInfo.h" > +#include "llvm/ADT/BitVector.h" > +#include "llvm/ADT/SmallSet.h" > + > +namespace llvm { > + class CriticalAntiDepBreaker : public AntiDepBreaker { > + MachineFunction& MF; > + MachineRegisterInfo &MRI; > + const TargetRegisterInfo *TRI; > + > + /// AllocatableSet - The set of allocatable registers. > + /// We'll be ignoring anti-dependencies on non-allocatable registers, > + /// because they may not be safe to break. > + const BitVector AllocatableSet; > + > + /// Classes - For live regs that are only used in one register class in a > + /// live range, the register class. If the register is not live, the > + /// corresponding value is null. If the register is live but used in > + /// multiple register classes, the corresponding value is -1 casted to a > + /// pointer. > + const TargetRegisterClass * > + Classes[TargetRegisterInfo::FirstVirtualRegister]; > + > + /// RegRegs - Map registers to all their references within a live range. > + std::multimap RegRefs; > + > + /// KillIndices - The index of the most recent kill (proceding bottom-up), > + /// or ~0u if the register is not live. > + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; > + > + /// DefIndices - The index of the most recent complete def (proceding bottom > + /// up), or ~0u if the register is live. > + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; > + > + /// KeepRegs - A set of registers which are live and cannot be changed to > + /// break anti-dependencies. > + SmallSet KeepRegs; > + > + public: > + CriticalAntiDepBreaker(MachineFunction& MFi); > + ~CriticalAntiDepBreaker(); > + > + /// Start - Initialize anti-dep breaking for a new basic block. > + void StartBlock(MachineBasicBlock *BB); > + > + /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path > + /// of the ScheduleDAG and break them by renaming registers. > + /// > + unsigned BreakAntiDependencies(std::vector& SUnits, > + MachineBasicBlock::iterator& Begin, > + MachineBasicBlock::iterator& End, > + unsigned InsertPosIndex); > + > + /// Observe - Update liveness information to account for the current > + /// instruction, which will not be scheduled. > + /// > + void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); > + > + /// Finish - Finish anti-dep breaking for a basic block. > + void FinishBlock(); > + > + private: > + void PrescanInstruction(MachineInstr *MI); > + void ScanInstruction(MachineInstr *MI, unsigned Count); > + unsigned findSuitableFreeRegister(unsigned AntiDepReg, > + unsigned LastNewReg, > + const TargetRegisterClass *); > + }; > +} > + > +#endif > > Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85127&r1=85126&r2=85127&view=diff > > ============================================================================== > --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) > +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 11:59:04 2009 > @@ -19,6 +19,7 @@ > //===----------------------------------------------------------------------===// > > #define DEBUG_TYPE "post-RA-sched" > +#include "CriticalAntiDepBreaker.h" > #include "ExactHazardRecognizer.h" > #include "SimpleHazardRecognizer.h" > #include "ScheduleDAGInstrs.h" > @@ -40,6 +41,7 @@ > #include "llvm/Support/Debug.h" > #include "llvm/Support/ErrorHandling.h" > #include "llvm/Support/raw_ostream.h" > +#include "llvm/ADT/BitVector.h" > #include "llvm/ADT/Statistic.h" > #include > #include > @@ -47,6 +49,7 @@ > > STATISTIC(NumNoops, "Number of noops inserted"); > STATISTIC(NumStalls, "Number of pipeline stalls"); > +STATISTIC(NumFixedAnti, "Number of fixed anti-dependencies"); > > // Post-RA scheduling is enabled with > // TargetSubtarget.enablePostRAScheduler(). This flag can be used to > @@ -55,10 +58,11 @@ > EnablePostRAScheduler("post-RA-scheduler", > cl::desc("Enable scheduling after register allocation"), > cl::init(false), cl::Hidden); > -static cl::opt > +static cl::opt > EnableAntiDepBreaking("break-anti-dependencies", > - cl::desc("Break post-RA scheduling anti-dependencies"), > - cl::init(true), cl::Hidden); > + cl::desc("Break post-RA scheduling anti-dependencies: " > + "\"critical\", \"all\", or \"none\""), > + cl::init("none"), cl::Hidden); > static cl::opt > EnablePostRAHazardAvoidance("avoid-hazards", > cl::desc("Enable exact hazard avoidance"), > @@ -116,56 +120,30 @@ > /// Topo - A topological ordering for SUnits. > ScheduleDAGTopologicalSort Topo; > > - /// AllocatableSet - The set of allocatable registers. > - /// We'll be ignoring anti-dependencies on non-allocatable registers, > - /// because they may not be safe to break. > - const BitVector AllocatableSet; > - > /// HazardRec - The hazard recognizer to use. > ScheduleHazardRecognizer *HazardRec; > > + /// AntiDepBreak - Anti-dependence breaking object, or NULL if none > + AntiDepBreaker *AntiDepBreak; > + > /// AA - AliasAnalysis for making memory reference queries. > AliasAnalysis *AA; > > - /// AntiDepMode - Anti-dependence breaking mode > - TargetSubtarget::AntiDepBreakMode AntiDepMode; > - > - /// Classes - For live regs that are only used in one register class in a > - /// live range, the register class. If the register is not live, the > - /// corresponding value is null. If the register is live but used in > - /// multiple register classes, the corresponding value is -1 casted to a > - /// pointer. > - const TargetRegisterClass * > - Classes[TargetRegisterInfo::FirstVirtualRegister]; > - > - /// RegRegs - Map registers to all their references within a live range. > - std::multimap RegRefs; > - > /// KillIndices - The index of the most recent kill (proceding bottom-up), > /// or ~0u if the register is not live. > unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; > > - /// DefIndices - The index of the most recent complete def (proceding bottom > - /// up), or ~0u if the register is live. > - unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; > - > - /// KeepRegs - A set of registers which are live and cannot be changed to > - /// break anti-dependencies. > - SmallSet KeepRegs; > - > public: > SchedulePostRATDList(MachineFunction &MF, > const MachineLoopInfo &MLI, > const MachineDominatorTree &MDT, > ScheduleHazardRecognizer *HR, > - AliasAnalysis *aa, > - TargetSubtarget::AntiDepBreakMode adm) > + AntiDepBreaker *ADB, > + AliasAnalysis *aa) > : ScheduleDAGInstrs(MF, MLI, MDT), Topo(SUnits), > - AllocatableSet(TRI->getAllocatableSet(MF)), > - HazardRec(HR), AA(aa), AntiDepMode(adm) {} > + HazardRec(HR), AntiDepBreak(ADB), AA(aa) {} > > ~SchedulePostRATDList() { > - delete HazardRec; > } > > /// StartBlock - Initialize register live-range state for scheduling in > @@ -177,11 +155,6 @@ > /// > void Schedule(); > > - /// FixupKills - Fix register kill flags that have been made > - /// invalid due to scheduling > - /// > - void FixupKills(MachineBasicBlock *MBB); > - > /// Observe - Update liveness information to account for the current > /// instruction, which will not be scheduled. > /// > @@ -191,17 +164,16 @@ > /// > void FinishBlock(); > > + /// FixupKills - Fix register kill flags that have been made > + /// invalid due to scheduling > + /// > + void FixupKills(MachineBasicBlock *MBB); > + > private: > - void PrescanInstruction(MachineInstr *MI); > - void ScanInstruction(MachineInstr *MI, unsigned Count); > void ReleaseSucc(SUnit *SU, SDep *SuccEdge); > void ReleaseSuccessors(SUnit *SU); > void ScheduleNodeTopDown(SUnit *SU, unsigned CurCycle); > void ListScheduleTopDown(); > - bool BreakAntiDependencies(); > - unsigned findSuitableFreeRegister(unsigned AntiDepReg, > - unsigned LastNewReg, > - const TargetRegisterClass *); > void StartBlockForKills(MachineBasicBlock *BB); > > // ToggleKillFlag - Toggle a register operand kill flag. Other > @@ -250,8 +222,9 @@ > > // Check for antidep breaking override... > if (EnableAntiDepBreaking.getPosition() > 0) { > - AntiDepMode = (EnableAntiDepBreaking) ? > - TargetSubtarget::ANTIDEP_CRITICAL : TargetSubtarget::ANTIDEP_NONE; > + AntiDepMode = (EnableAntiDepBreaking == "all") ? TargetSubtarget::ANTIDEP_ALL : > + (EnableAntiDepBreaking == "critical") ? TargetSubtarget::ANTIDEP_CRITICAL : > + TargetSubtarget::ANTIDEP_NONE; > } > > DEBUG(errs() << "PostRAScheduler\n"); > @@ -262,8 +235,12 @@ > ScheduleHazardRecognizer *HR = EnablePostRAHazardAvoidance ? > (ScheduleHazardRecognizer *)new ExactHazardRecognizer(InstrItins) : > (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); > + AntiDepBreaker *ADB = > + (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME */ : > + (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? > + new CriticalAntiDepBreaker(Fn) : NULL; > > - SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, AA, AntiDepMode); > + SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); > > // Loop over all of the basic blocks > for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end(); > @@ -311,6 +288,9 @@ > Scheduler.FixupKills(MBB); > } > > + delete HR; > + delete ADB; > + > return true; > } > > @@ -321,78 +301,10 @@ > // Call the superclass. > ScheduleDAGInstrs::StartBlock(BB); > > - // Reset the hazard recognizer. > + // Reset the hazard recognizer and anti-dep breaker. > HazardRec->Reset(); > - > - // Clear out the register class data. > - std::fill(Classes, array_endof(Classes), > - static_cast(0)); > - > - // Initialize the indices to indicate that no registers are live. > - std::fill(KillIndices, array_endof(KillIndices), ~0u); > - std::fill(DefIndices, array_endof(DefIndices), BB->size()); > - > - // Clear "do not change" set. > - KeepRegs.clear(); > - > - bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); > - > - // Determine the live-out physregs for this block. > - if (IsReturnBlock) { > - // In a return block, examine the function live-out regs. > - for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), > - E = MRI.liveout_end(); I != E; ++I) { > - unsigned Reg = *I; > - Classes[Reg] = reinterpret_cast(-1); > - KillIndices[Reg] = BB->size(); > - DefIndices[Reg] = ~0u; > - // Repeat, for all aliases. > - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > - unsigned AliasReg = *Alias; > - Classes[AliasReg] = reinterpret_cast(-1); > - KillIndices[AliasReg] = BB->size(); > - DefIndices[AliasReg] = ~0u; > - } > - } > - } else { > - // In a non-return block, examine the live-in regs of all successors. > - for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), > - SE = BB->succ_end(); SI != SE; ++SI) > - for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), > - E = (*SI)->livein_end(); I != E; ++I) { > - unsigned Reg = *I; > - Classes[Reg] = reinterpret_cast(-1); > - KillIndices[Reg] = BB->size(); > - DefIndices[Reg] = ~0u; > - // Repeat, for all aliases. > - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > - unsigned AliasReg = *Alias; > - Classes[AliasReg] = reinterpret_cast(-1); > - KillIndices[AliasReg] = BB->size(); > - DefIndices[AliasReg] = ~0u; > - } > - } > - } > - > - // Mark live-out callee-saved registers. In a return block this is > - // all callee-saved registers. In non-return this is any > - // callee-saved register that is not saved in the prolog. > - const MachineFrameInfo *MFI = MF.getFrameInfo(); > - BitVector Pristine = MFI->getPristineRegs(BB); > - for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { > - unsigned Reg = *I; > - if (!IsReturnBlock && !Pristine.test(Reg)) continue; > - Classes[Reg] = reinterpret_cast(-1); > - KillIndices[Reg] = BB->size(); > - DefIndices[Reg] = ~0u; > - // Repeat, for all aliases. > - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > - unsigned AliasReg = *Alias; > - Classes[AliasReg] = reinterpret_cast(-1); > - KillIndices[AliasReg] = BB->size(); > - DefIndices[AliasReg] = ~0u; > - } > - } > + if (AntiDepBreak != NULL) > + AntiDepBreak->StartBlock(BB); > } > > /// Schedule - Schedule the instruction range using list scheduling. > @@ -403,8 +315,11 @@ > // Build the scheduling graph. > BuildSchedGraph(AA); > > - if (AntiDepMode != TargetSubtarget::ANTIDEP_NONE) { > - if (BreakAntiDependencies()) { > + if (AntiDepBreak != NULL) { > + unsigned Broken = > + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, > + InsertPosIndex); > + if (Broken > 0) { > // We made changes. Update the dependency graph. > // Theoretically we could update the graph in place: > // When a live range is changed to use a different register, remove > @@ -415,6 +330,8 @@ > EntrySU = SUnit(); > ExitSU = SUnit(); > BuildSchedGraph(AA); > + > + NumFixedAnti += Broken; > } > } > > @@ -432,436 +349,20 @@ > /// instruction, which will not be scheduled. > /// > void SchedulePostRATDList::Observe(MachineInstr *MI, unsigned Count) { > - assert(Count < InsertPosIndex && "Instruction index out of expected range!"); > - > - // Any register which was defined within the previous scheduling region > - // may have been rescheduled and its lifetime may overlap with registers > - // in ways not reflected in our current liveness state. For each such > - // register, adjust the liveness state to be conservatively correct. > - for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) > - if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { > - assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); > - // Mark this register to be non-renamable. > - Classes[Reg] = reinterpret_cast(-1); > - // Move the def index to the end of the previous region, to reflect > - // that the def could theoretically have been scheduled at the end. > - DefIndices[Reg] = InsertPosIndex; > - } > - > - PrescanInstruction(MI); > - ScanInstruction(MI, Count); > + if (AntiDepBreak != NULL) > + AntiDepBreak->Observe(MI, Count, InsertPosIndex); > } > > /// FinishBlock - Clean up register live-range state. > /// > void SchedulePostRATDList::FinishBlock() { > - RegRefs.clear(); > + if (AntiDepBreak != NULL) > + AntiDepBreak->FinishBlock(); > > // Call the superclass. > ScheduleDAGInstrs::FinishBlock(); > } > > -/// CriticalPathStep - Return the next SUnit after SU on the bottom-up > -/// critical path. > -static SDep *CriticalPathStep(SUnit *SU) { > - SDep *Next = 0; > - unsigned NextDepth = 0; > - // Find the predecessor edge with the greatest depth. > - for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); > - P != PE; ++P) { > - SUnit *PredSU = P->getSUnit(); > - unsigned PredLatency = P->getLatency(); > - unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; > - // In the case of a latency tie, prefer an anti-dependency edge over > - // other types of edges. > - if (NextDepth < PredTotalLatency || > - (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { > - NextDepth = PredTotalLatency; > - Next = &*P; > - } > - } > - return Next; > -} > - > -void SchedulePostRATDList::PrescanInstruction(MachineInstr *MI) { > - // Scan the register operands for this instruction and update > - // Classes and RegRefs. > - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > - MachineOperand &MO = MI->getOperand(i); > - if (!MO.isReg()) continue; > - unsigned Reg = MO.getReg(); > - if (Reg == 0) continue; > - const TargetRegisterClass *NewRC = 0; > - > - if (i < MI->getDesc().getNumOperands()) > - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); > - > - // For now, only allow the register to be changed if its register > - // class is consistent across all uses. > - if (!Classes[Reg] && NewRC) > - Classes[Reg] = NewRC; > - else if (!NewRC || Classes[Reg] != NewRC) > - Classes[Reg] = reinterpret_cast(-1); > - > - // Now check for aliases. > - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > - // If an alias of the reg is used during the live range, give up. > - // Note that this allows us to skip checking if AntiDepReg > - // overlaps with any of the aliases, among other things. > - unsigned AliasReg = *Alias; > - if (Classes[AliasReg]) { > - Classes[AliasReg] = reinterpret_cast(-1); > - Classes[Reg] = reinterpret_cast(-1); > - } > - } > - > - // If we're still willing to consider this register, note the reference. > - if (Classes[Reg] != reinterpret_cast(-1)) > - RegRefs.insert(std::make_pair(Reg, &MO)); > - > - // It's not safe to change register allocation for source operands of > - // that have special allocation requirements. > - if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { > - if (KeepRegs.insert(Reg)) { > - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); > - *Subreg; ++Subreg) > - KeepRegs.insert(*Subreg); > - } > - } > - } > -} > - > -void SchedulePostRATDList::ScanInstruction(MachineInstr *MI, > - unsigned Count) { > - // Update liveness. > - // Proceding upwards, registers that are defed but not used in this > - // instruction are now dead. > - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > - MachineOperand &MO = MI->getOperand(i); > - if (!MO.isReg()) continue; > - unsigned Reg = MO.getReg(); > - if (Reg == 0) continue; > - if (!MO.isDef()) continue; > - // Ignore two-addr defs. > - if (MI->isRegTiedToUseOperand(i)) continue; > - > - DefIndices[Reg] = Count; > - KillIndices[Reg] = ~0u; > - assert(((KillIndices[Reg] == ~0u) != > - (DefIndices[Reg] == ~0u)) && > - "Kill and Def maps aren't consistent for Reg!"); > - KeepRegs.erase(Reg); > - Classes[Reg] = 0; > - RegRefs.erase(Reg); > - // Repeat, for all subregs. > - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); > - *Subreg; ++Subreg) { > - unsigned SubregReg = *Subreg; > - DefIndices[SubregReg] = Count; > - KillIndices[SubregReg] = ~0u; > - KeepRegs.erase(SubregReg); > - Classes[SubregReg] = 0; > - RegRefs.erase(SubregReg); > - } > - // Conservatively mark super-registers as unusable. > - for (const unsigned *Super = TRI->getSuperRegisters(Reg); > - *Super; ++Super) { > - unsigned SuperReg = *Super; > - Classes[SuperReg] = reinterpret_cast(-1); > - } > - } > - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > - MachineOperand &MO = MI->getOperand(i); > - if (!MO.isReg()) continue; > - unsigned Reg = MO.getReg(); > - if (Reg == 0) continue; > - if (!MO.isUse()) continue; > - > - const TargetRegisterClass *NewRC = 0; > - if (i < MI->getDesc().getNumOperands()) > - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); > - > - // For now, only allow the register to be changed if its register > - // class is consistent across all uses. > - if (!Classes[Reg] && NewRC) > - Classes[Reg] = NewRC; > - else if (!NewRC || Classes[Reg] != NewRC) > - Classes[Reg] = reinterpret_cast(-1); > - > - RegRefs.insert(std::make_pair(Reg, &MO)); > - > - // It wasn't previously live but now it is, this is a kill. > - if (KillIndices[Reg] == ~0u) { > - KillIndices[Reg] = Count; > - DefIndices[Reg] = ~0u; > - assert(((KillIndices[Reg] == ~0u) != > - (DefIndices[Reg] == ~0u)) && > - "Kill and Def maps aren't consistent for Reg!"); > - } > - // Repeat, for all aliases. > - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { > - unsigned AliasReg = *Alias; > - if (KillIndices[AliasReg] == ~0u) { > - KillIndices[AliasReg] = Count; > - DefIndices[AliasReg] = ~0u; > - } > - } > - } > -} > - > -unsigned > -SchedulePostRATDList::findSuitableFreeRegister(unsigned AntiDepReg, > - unsigned LastNewReg, > - const TargetRegisterClass *RC) { > - for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), > - RE = RC->allocation_order_end(MF); R != RE; ++R) { > - unsigned NewReg = *R; > - // Don't replace a register with itself. > - if (NewReg == AntiDepReg) continue; > - // Don't replace a register with one that was recently used to repair > - // an anti-dependence with this AntiDepReg, because that would > - // re-introduce that anti-dependence. > - if (NewReg == LastNewReg) continue; > - // If NewReg is dead and NewReg's most recent def is not before > - // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. > - assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && > - "Kill and Def maps aren't consistent for AntiDepReg!"); > - assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && > - "Kill and Def maps aren't consistent for NewReg!"); > - if (KillIndices[NewReg] != ~0u || > - Classes[NewReg] == reinterpret_cast(-1) || > - KillIndices[AntiDepReg] > DefIndices[NewReg]) > - continue; > - return NewReg; > - } > - > - // No registers are free and available! > - return 0; > -} > - > -/// BreakAntiDependencies - Identifiy anti-dependencies along the critical path > -/// of the ScheduleDAG and break them by renaming registers. > -/// > -bool SchedulePostRATDList::BreakAntiDependencies() { > - // The code below assumes that there is at least one instruction, > - // so just duck out immediately if the block is empty. > - if (SUnits.empty()) return false; > - > - // Find the node at the bottom of the critical path. > - SUnit *Max = 0; > - for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { > - SUnit *SU = &SUnits[i]; > - if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) > - Max = SU; > - } > - > -#ifndef NDEBUG > - { > - DEBUG(errs() << "Critical path has total latency " > - << (Max->getDepth() + Max->Latency) << "\n"); > - DEBUG(errs() << "Available regs:"); > - for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { > - if (KillIndices[Reg] == ~0u) > - DEBUG(errs() << " " << TRI->getName(Reg)); > - } > - DEBUG(errs() << '\n'); > - } > -#endif > - > - // Track progress along the critical path through the SUnit graph as we walk > - // the instructions. > - SUnit *CriticalPathSU = Max; > - MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); > - > - // Consider this pattern: > - // A = ... > - // ... = A > - // A = ... > - // ... = A > - // A = ... > - // ... = A > - // A = ... > - // ... = A > - // There are three anti-dependencies here, and without special care, > - // we'd break all of them using the same register: > - // A = ... > - // ... = A > - // B = ... > - // ... = B > - // B = ... > - // ... = B > - // B = ... > - // ... = B > - // because at each anti-dependence, B is the first register that > - // isn't A which is free. This re-introduces anti-dependencies > - // at all but one of the original anti-dependencies that we were > - // trying to break. To avoid this, keep track of the most recent > - // register that each register was replaced with, avoid > - // using it to repair an anti-dependence on the same register. > - // This lets us produce this: > - // A = ... > - // ... = A > - // B = ... > - // ... = B > - // C = ... > - // ... = C > - // B = ... > - // ... = B > - // This still has an anti-dependence on B, but at least it isn't on the > - // original critical path. > - // > - // TODO: If we tracked more than one register here, we could potentially > - // fix that remaining critical edge too. This is a little more involved, > - // because unlike the most recent register, less recent registers should > - // still be considered, though only if no other registers are available. > - unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; > - > - // Attempt to break anti-dependence edges on the critical path. Walk the > - // instructions from the bottom up, tracking information about liveness > - // as we go to help determine which registers are available. > - bool Changed = false; > - unsigned Count = InsertPosIndex - 1; > - for (MachineBasicBlock::iterator I = InsertPos, E = Begin; > - I != E; --Count) { > - MachineInstr *MI = --I; > - > - // Check if this instruction has a dependence on the critical path that > - // is an anti-dependence that we may be able to break. If it is, set > - // AntiDepReg to the non-zero register associated with the anti-dependence. > - // > - // We limit our attention to the critical path as a heuristic to avoid > - // breaking anti-dependence edges that aren't going to significantly > - // impact the overall schedule. There are a limited number of registers > - // and we want to save them for the important edges. > - // > - // TODO: Instructions with multiple defs could have multiple > - // anti-dependencies. The current code here only knows how to break one > - // edge per instruction. Note that we'd have to be able to break all of > - // the anti-dependencies in an instruction in order to be effective. > - unsigned AntiDepReg = 0; > - if (MI == CriticalPathMI) { > - if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { > - SUnit *NextSU = Edge->getSUnit(); > - > - // Only consider anti-dependence edges. > - if (Edge->getKind() == SDep::Anti) { > - AntiDepReg = Edge->getReg(); > - assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); > - if (!AllocatableSet.test(AntiDepReg)) > - // Don't break anti-dependencies on non-allocatable registers. > - AntiDepReg = 0; > - else if (KeepRegs.count(AntiDepReg)) > - // Don't break anti-dependencies if an use down below requires > - // this exact register. > - AntiDepReg = 0; > - else { > - // If the SUnit has other dependencies on the SUnit that it > - // anti-depends on, don't bother breaking the anti-dependency > - // since those edges would prevent such units from being > - // scheduled past each other regardless. > - // > - // Also, if there are dependencies on other SUnits with the > - // same register as the anti-dependency, don't attempt to > - // break it. > - for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), > - PE = CriticalPathSU->Preds.end(); P != PE; ++P) > - if (P->getSUnit() == NextSU ? > - (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : > - (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { > - AntiDepReg = 0; > - break; > - } > - } > - } > - CriticalPathSU = NextSU; > - CriticalPathMI = CriticalPathSU->getInstr(); > - } else { > - // We've reached the end of the critical path. > - CriticalPathSU = 0; > - CriticalPathMI = 0; > - } > - } > - > - PrescanInstruction(MI); > - > - if (MI->getDesc().hasExtraDefRegAllocReq()) > - // If this instruction's defs have special allocation requirement, don't > - // break this anti-dependency. > - AntiDepReg = 0; > - else if (AntiDepReg) { > - // If this instruction has a use of AntiDepReg, breaking it > - // is invalid. > - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { > - MachineOperand &MO = MI->getOperand(i); > - if (!MO.isReg()) continue; > - unsigned Reg = MO.getReg(); > - if (Reg == 0) continue; > - if (MO.isUse() && AntiDepReg == Reg) { > - AntiDepReg = 0; > - break; > - } > - } > - } > - > - // Determine AntiDepReg's register class, if it is live and is > - // consistently used within a single class. > - const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; > - assert((AntiDepReg == 0 || RC != NULL) && > - "Register should be live if it's causing an anti-dependence!"); > - if (RC == reinterpret_cast(-1)) > - AntiDepReg = 0; > - > - // Look for a suitable register to use to break the anti-depenence. > - // > - // TODO: Instead of picking the first free register, consider which might > - // be the best. > - if (AntiDepReg != 0) { > - if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, > - LastNewReg[AntiDepReg], > - RC)) { > - DEBUG(errs() << "Breaking anti-dependence edge on " > - << TRI->getName(AntiDepReg) > - << " with " << RegRefs.count(AntiDepReg) << " references" > - << " using " << TRI->getName(NewReg) << "!\n"); > - > - // Update the references to the old register to refer to the new > - // register. > - std::pair::iterator, > - std::multimap::iterator> > - Range = RegRefs.equal_range(AntiDepReg); > - for (std::multimap::iterator > - Q = Range.first, QE = Range.second; Q != QE; ++Q) > - Q->second->setReg(NewReg); > - > - // We just went back in time and modified history; the > - // liveness information for the anti-depenence reg is now > - // inconsistent. Set the state as if it were dead. > - Classes[NewReg] = Classes[AntiDepReg]; > - DefIndices[NewReg] = DefIndices[AntiDepReg]; > - KillIndices[NewReg] = KillIndices[AntiDepReg]; > - assert(((KillIndices[NewReg] == ~0u) != > - (DefIndices[NewReg] == ~0u)) && > - "Kill and Def maps aren't consistent for NewReg!"); > - > - Classes[AntiDepReg] = 0; > - DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; > - KillIndices[AntiDepReg] = ~0u; > - assert(((KillIndices[AntiDepReg] == ~0u) != > - (DefIndices[AntiDepReg] == ~0u)) && > - "Kill and Def maps aren't consistent for AntiDepReg!"); > - > - RegRefs.erase(AntiDepReg); > - Changed = true; > - LastNewReg[AntiDepReg] = NewReg; > - } > - } > - > - ScanInstruction(MI, Count); > - } > - > - return Changed; > -} > - > /// StartBlockForKills - Initialize register live-range state for updating kills > /// > void SchedulePostRATDList::StartBlockForKills(MachineBasicBlock *BB) { > > Modified: llvm/trunk/lib/Target/ARM/ARMSubtarget.h > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMSubtarget.h?rev=85127&r1=85126&r2=85127&view=diff > > ============================================================================== > --- llvm/trunk/lib/Target/ARM/ARMSubtarget.h (original) > +++ llvm/trunk/lib/Target/ARM/ARMSubtarget.h Mon Oct 26 11:59:04 2009 > @@ -130,7 +130,7 @@ > /// for Thumb1. > bool enablePostRAScheduler(CodeGenOpt::Level OptLevel, > TargetSubtarget::AntiDepBreakMode& mode) const { > - mode = TargetSubtarget::ANTIDEP_NONE; > + mode = TargetSubtarget::ANTIDEP_CRITICAL; > return PostRAScheduler && OptLevel >= CodeGenOpt::Default; > } > > > Modified: llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll?rev=85127&r1=85126&r2=85127&view=diff > > ============================================================================== > --- llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll (original) > +++ llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Mon Oct 26 11:59:04 2009 > @@ -1,7 +1,7 @@ > -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=false > %t > +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=none > %t > ; RUN: grep {%xmm0} %t | count 14 > ; RUN: not grep {%xmm1} %t > -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies > %t > +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=critical > %t > ; RUN: grep {%xmm0} %t | count 7 > ; RUN: grep {%xmm1} %t | count 7 > > > > _______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits From clattner at apple.com Mon Oct 26 16:26:16 2009 From: clattner at apple.com (Chris Lattner) Date: Mon, 26 Oct 2009 14:26:16 -0700 Subject: [llvm-commits] [llvm] r85147 - in /llvm/trunk: Makefile.config.in autoconf/configure.ac autoconf/m4/cxx_flag_check.m4 In-Reply-To: <200910261958.n9QJwjjp000406@zion.cs.uiuc.edu> References: <200910261958.n9QJwjjp000406@zion.cs.uiuc.edu> Message-ID: <5E72AE8E-1A5B-4A67-A916-CE8ABEC0D83E@apple.com> On Oct 26, 2009, at 12:58 PM, Julien Lerouge wrote: > Author: jlerouge > Date: Mon Oct 26 14:58:44 2009 > New Revision: 85147 > > URL: http://llvm.org/viewvc/llvm-project?rev=85147&view=rev > Log: > Add an autoconf test to check for optional compiler flags like > -Wno-missing-field-initializers or -Wno-variadic-macros. Nice, thanks Julien! -Chris > > Added: > llvm/trunk/autoconf/m4/cxx_flag_check.m4 > Modified: > llvm/trunk/Makefile.config.in > llvm/trunk/autoconf/configure.ac > > Modified: llvm/trunk/Makefile.config.in > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/Makefile.config.in?rev=85147&r1=85146&r2=85147&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/Makefile.config.in (original) > +++ llvm/trunk/Makefile.config.in Mon Oct 26 14:58:44 2009 > @@ -320,3 +320,9 @@ > # support (via the -load option). > ENABLE_LLVMC_DYNAMIC_PLUGINS = 1 > #@ENABLE_LLVMC_DYNAMIC_PLUGINS@ > + > +# Optional flags supported by the compiler > +# -Wno-missing-field-initializers > +NO_MISSING_FIELD_INITIALIZERS = @NO_MISSING_FIELD_INITIALIZERS@ > +# -Wno-variadic-macros > +NO_VARIADIC_MACROS = @NO_VARIADIC_MACROS@ > > Modified: llvm/trunk/autoconf/configure.ac > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/autoconf/configure.ac?rev=85147&r1=85146&r2=85147&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/autoconf/configure.ac (original) > +++ llvm/trunk/autoconf/configure.ac Mon Oct 26 14:58:44 2009 > @@ -943,6 +943,12 @@ > dnl Tool compatibility is okay if we make it here. > AC_MSG_RESULT([ok]) > > +dnl Check optional compiler flags. > +AC_MSG_CHECKING([optional compiler flags]) > +CXX_FLAG_CHECK(NO_VARIADIC_MACROS, [-Wno-variadic-macros]) > +CXX_FLAG_CHECK(NO_MISSING_FIELD_INITIALIZERS, [-Wno-missing-field- > initializers]) > +AC_MSG_RESULT([$NO_VARIADIC_MACROS $NO_MISSING_FIELD_INITIALIZERS]) > + > dnl > = > = > = > -----------------------------------------------------------------------= > == > dnl=== > dnl=== SECTION 5: Check for libraries > > Added: llvm/trunk/autoconf/m4/cxx_flag_check.m4 > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/autoconf/m4/cxx_flag_check.m4?rev=85147&view=auto > > = > = > = > = > = > = > = > = > ====================================================================== > --- llvm/trunk/autoconf/m4/cxx_flag_check.m4 (added) > +++ llvm/trunk/autoconf/m4/cxx_flag_check.m4 Mon Oct 26 14:58:44 2009 > @@ -0,0 +1,2 @@ > +AC_DEFUN([CXX_FLAG_CHECK], > + [AC_SUBST($1, `$CXX $2 -fsyntax-only -xc /dev/null 2>/dev/null && > echo $2`)]) > > > _______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits From david_goodwin at apple.com Mon Oct 26 16:27:07 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 14:27:07 -0700 Subject: [llvm-commits] [llvm] r85127 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/CMakeLists.txt lib/CodeGen/CriticalAntiDepBreaker.cpp lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp lib/Target/ARM/ARMSubtarget.h test/CodeGen/X86/break-anti-dependencies.ll In-Reply-To: References: <200910261659.n9QGx4Yj023936@zion.cs.uiuc.edu> Message-ID: <35E02EF7-4CBB-4607-BC71-849E46D366D2@apple.com> I was just following the style of ScheduleHazardRecognizer, as it has the same "visibility" as AntiDepBreaker. But I can move it if that would be better... David On Oct 26, 2009, at 2:23 PM, Evan Cheng wrote: > It seems to be AntiDepBreaker.h should be inside lib/CodeGen. Is > there a reason to expose it? > > Evan > > On Oct 26, 2009, at 9:59 AM, David Goodwin wrote: > >> Author: david_goodwin >> Date: Mon Oct 26 11:59:04 2009 >> New Revision: 85127 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=85127&view=rev >> Log: >> Break anti-dependence breaking out into its own class. >> >> Added: >> llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >> Modified: >> llvm/trunk/lib/CodeGen/CMakeLists.txt >> llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >> llvm/trunk/lib/Target/ARM/ARMSubtarget.h >> llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >> >> Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (added) >> +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,56 @@ >> +//=- llvm/CodeGen/AntiDepBreaker.h - Anti-Dependence Breaking -*- C >> ++ -*-=// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the AntiDepBreaker class, which implements >> +// anti-dependence breaking heuristics for post-register- >> allocation scheduling. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#ifndef LLVM_CODEGEN_ANTIDEPBREAKER_H >> +#define LLVM_CODEGEN_ANTIDEPBREAKER_H >> + >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/CodeGen/MachineFunction.h" >> +#include "llvm/CodeGen/MachineRegisterInfo.h" >> +#include "llvm/CodeGen/ScheduleDAG.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> + >> +namespace llvm { >> + >> +/// AntiDepBreaker - This class works into conjunction with the >> +/// post-RA scheduler to rename registers to break register >> +/// anti-dependencies. >> +class AntiDepBreaker { >> +public: >> + /// Start - Initialize anti-dep breaking for a new basic block. >> + virtual void StartBlock(MachineBasicBlock *BB) =0; >> + >> + /// BreakAntiDependencies - Identifiy anti-dependencies within a >> + /// basic-block region and break them by renaming registers. >> Return >> + /// the number of anti-dependencies broken. >> + /// >> + virtual unsigned BreakAntiDependencies(std::vector& SUnits, >> + >> MachineBasicBlock::iterator& Begin, >> + >> MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex) >> =0; >> + >> + /// Observe - Update liveness information to account for the >> current >> + /// instruction, which will not be scheduled. >> + /// >> + virtual void Observe(MachineInstr *MI, unsigned Count, >> + unsigned InsertPosIndex) =0; >> + >> + /// Finish - Finish anti-dep breaking for a basic block. >> + virtual void FinishBlock() =0; >> +}; >> + >> +} >> + >> +#endif >> >> Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) >> +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 11:59:04 2009 >> @@ -1,6 +1,7 @@ >> add_llvm_library(LLVMCodeGen >> BranchFolding.cpp >> CodePlacementOpt.cpp >> + CriticalAntiDepBreaker.cpp >> DeadMachineInstructionElim.cpp >> DwarfEHPrepare.cpp >> ELFCodeEmitter.cpp >> >> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp (added) >> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,539 @@ >> +//===----- CriticalAntiDepBreaker.cpp - Anti-dep breaker -------- >> ---------===// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the CriticalAntiDepBreaker class, which >> +// implements register anti-dependence breaking along a blocks >> +// critical path during post-RA scheduler. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#define DEBUG_TYPE "critical-antidep" >> +#include "CriticalAntiDepBreaker.h" >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/Target/TargetMachine.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> +#include "llvm/Support/Debug.h" >> +#include "llvm/Support/ErrorHandling.h" >> +#include "llvm/Support/raw_ostream.h" >> + >> +using namespace llvm; >> + >> +CriticalAntiDepBreaker:: >> +CriticalAntiDepBreaker(MachineFunction& MFi) : >> + AntiDepBreaker(), MF(MFi), >> + MRI(MF.getRegInfo()), >> + TRI(MF.getTarget().getRegisterInfo()), >> + AllocatableSet(TRI->getAllocatableSet(MF)) >> +{ >> +} >> + >> +CriticalAntiDepBreaker::~CriticalAntiDepBreaker() { >> +} >> + >> +void CriticalAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { >> + // Clear out the register class data. >> + std::fill(Classes, array_endof(Classes), >> + static_cast(0)); >> + >> + // Initialize the indices to indicate that no registers are live. >> + std::fill(KillIndices, array_endof(KillIndices), ~0u); >> + std::fill(DefIndices, array_endof(DefIndices), BB->size()); >> + >> + // Clear "do not change" set. >> + KeepRegs.clear(); >> + >> + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc >> ().isReturn()); >> + >> + // Determine the live-out physregs for this block. >> + if (IsReturnBlock) { >> + // In a return block, examine the function live-out regs. >> + for (MachineRegisterInfo::liveout_iterator I = >> MRI.liveout_begin(), >> + E = MRI.liveout_end(); I != E; ++I) { >> + unsigned Reg = *I; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } else { >> + // In a non-return block, examine the live-in regs of all >> successors. >> + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >> + SE = BB->succ_end(); SI != SE; ++SI) >> + for (MachineBasicBlock::livein_iterator I = (*SI)- >> >livein_begin(), >> + E = (*SI)->livein_end(); I != E; ++I) { >> + unsigned Reg = *I; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); >> *Alias; ++Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast> *>(-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } >> + >> + // Mark live-out callee-saved registers. In a return block this is >> + // all callee-saved registers. In non-return this is any >> + // callee-saved register that is not saved in the prolog. >> + const MachineFrameInfo *MFI = MF.getFrameInfo(); >> + BitVector Pristine = MFI->getPristineRegs(BB); >> + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >> + unsigned Reg = *I; >> + if (!IsReturnBlock && !Pristine.test(Reg)) continue; >> + Classes[Reg] = reinterpret_cast(-1); >> + KillIndices[Reg] = BB->size(); >> + DefIndices[Reg] = ~0u; >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + KillIndices[AliasReg] = BB->size(); >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> +} >> + >> +void CriticalAntiDepBreaker::FinishBlock() { >> + RegRefs.clear(); >> + KeepRegs.clear(); >> +} >> + >> +void CriticalAntiDepBreaker::Observe(MachineInstr *MI, unsigned >> Count, >> + unsigned InsertPosIndex) { >> + assert(Count < InsertPosIndex && "Instruction index out of >> expected range!"); >> + >> + // Any register which was defined within the previous scheduling >> region >> + // may have been rescheduled and its lifetime may overlap with >> registers >> + // in ways not reflected in our current liveness state. For each >> such >> + // register, adjust the liveness state to be conservatively >> correct. >> + for (unsigned Reg = 0; Reg != >> TargetRegisterInfo::FirstVirtualRegister; ++Reg) >> + if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= >> Count) { >> + assert(KillIndices[Reg] == ~0u && "Clobbered register is >> live!"); >> + // Mark this register to be non-renamable. >> + Classes[Reg] = reinterpret_cast(-1); >> + // Move the def index to the end of the previous region, to >> reflect >> + // that the def could theoretically have been scheduled at >> the end. >> + DefIndices[Reg] = InsertPosIndex; >> + } >> + >> + PrescanInstruction(MI); >> + ScanInstruction(MI, Count); >> +} >> + >> +/// CriticalPathStep - Return the next SUnit after SU on the >> bottom-up >> +/// critical path. >> +static SDep *CriticalPathStep(SUnit *SU) { >> + SDep *Next = 0; >> + unsigned NextDepth = 0; >> + // Find the predecessor edge with the greatest depth. >> + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU- >> >Preds.end(); >> + P != PE; ++P) { >> + SUnit *PredSU = P->getSUnit(); >> + unsigned PredLatency = P->getLatency(); >> + unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >> + // In the case of a latency tie, prefer an anti-dependency >> edge over >> + // other types of edges. >> + if (NextDepth < PredTotalLatency || >> + (NextDepth == PredTotalLatency && P->getKind() == >> SDep::Anti)) { >> + NextDepth = PredTotalLatency; >> + Next = &*P; >> + } >> + } >> + return Next; >> +} >> + >> +void CriticalAntiDepBreaker::PrescanInstruction(MachineInstr *MI) { >> + // Scan the register operands for this instruction and update >> + // Classes and RegRefs. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + const TargetRegisterClass *NewRC = 0; >> + >> + if (i < MI->getDesc().getNumOperands()) >> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> + >> + // For now, only allow the register to be changed if its >> register >> + // class is consistent across all uses. >> + if (!Classes[Reg] && NewRC) >> + Classes[Reg] = NewRC; >> + else if (!NewRC || Classes[Reg] != NewRC) >> + Classes[Reg] = reinterpret_cast(-1); >> + >> + // Now check for aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + // If an alias of the reg is used during the live range, >> give up. >> + // Note that this allows us to skip checking if AntiDepReg >> + // overlaps with any of the aliases, among other things. >> + unsigned AliasReg = *Alias; >> + if (Classes[AliasReg]) { >> + Classes[AliasReg] = reinterpret_cast >> (-1); >> + Classes[Reg] = reinterpret_cast(-1); >> + } >> + } >> + >> + // If we're still willing to consider this register, note the >> reference. >> + if (Classes[Reg] != reinterpret_cast(-1)) >> + RegRefs.insert(std::make_pair(Reg, &MO)); >> + >> + // It's not safe to change register allocation for source >> operands of >> + // that have special allocation requirements. >> + if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >> + if (KeepRegs.insert(Reg)) { >> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> + *Subreg; ++Subreg) >> + KeepRegs.insert(*Subreg); >> + } >> + } >> + } >> +} >> + >> +void CriticalAntiDepBreaker::ScanInstruction(MachineInstr *MI, >> + unsigned Count) { >> + // Update liveness. >> + // Proceding upwards, registers that are defed but not used in >> this >> + // instruction are now dead. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (!MO.isDef()) continue; >> + // Ignore two-addr defs. >> + if (MI->isRegTiedToUseOperand(i)) continue; >> + >> + DefIndices[Reg] = Count; >> + KillIndices[Reg] = ~0u; >> + assert(((KillIndices[Reg] == ~0u) != >> + (DefIndices[Reg] == ~0u)) && >> + "Kill and Def maps aren't consistent for Reg!"); >> + KeepRegs.erase(Reg); >> + Classes[Reg] = 0; >> + RegRefs.erase(Reg); >> + // Repeat, for all subregs. >> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> + *Subreg; ++Subreg) { >> + unsigned SubregReg = *Subreg; >> + DefIndices[SubregReg] = Count; >> + KillIndices[SubregReg] = ~0u; >> + KeepRegs.erase(SubregReg); >> + Classes[SubregReg] = 0; >> + RegRefs.erase(SubregReg); >> + } >> + // Conservatively mark super-registers as unusable. >> + for (const unsigned *Super = TRI->getSuperRegisters(Reg); >> + *Super; ++Super) { >> + unsigned SuperReg = *Super; >> + Classes[SuperReg] = reinterpret_cast >> (-1); >> + } >> + } >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (!MO.isUse()) continue; >> + >> + const TargetRegisterClass *NewRC = 0; >> + if (i < MI->getDesc().getNumOperands()) >> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> + >> + // For now, only allow the register to be changed if its >> register >> + // class is consistent across all uses. >> + if (!Classes[Reg] && NewRC) >> + Classes[Reg] = NewRC; >> + else if (!NewRC || Classes[Reg] != NewRC) >> + Classes[Reg] = reinterpret_cast(-1); >> + >> + RegRefs.insert(std::make_pair(Reg, &MO)); >> + >> + // It wasn't previously live but now it is, this is a kill. >> + if (KillIndices[Reg] == ~0u) { >> + KillIndices[Reg] = Count; >> + DefIndices[Reg] = ~0u; >> + assert(((KillIndices[Reg] == ~0u) != >> + (DefIndices[Reg] == ~0u)) && >> + "Kill and Def maps aren't consistent for Reg!"); >> + } >> + // Repeat, for all aliases. >> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> + unsigned AliasReg = *Alias; >> + if (KillIndices[AliasReg] == ~0u) { >> + KillIndices[AliasReg] = Count; >> + DefIndices[AliasReg] = ~0u; >> + } >> + } >> + } >> +} >> + >> +unsigned >> +CriticalAntiDepBreaker::findSuitableFreeRegister(unsigned >> AntiDepReg, >> + unsigned >> LastNewReg, >> + const >> TargetRegisterClass *RC) { >> + for (TargetRegisterClass::iterator R = RC->allocation_order_begin >> (MF), >> + RE = RC->allocation_order_end(MF); R != RE; ++R) { >> + unsigned NewReg = *R; >> + // Don't replace a register with itself. >> + if (NewReg == AntiDepReg) continue; >> + // Don't replace a register with one that was recently used to >> repair >> + // an anti-dependence with this AntiDepReg, because that would >> + // re-introduce that anti-dependence. >> + if (NewReg == LastNewReg) continue; >> + // If NewReg is dead and NewReg's most recent def is not before >> + // AntiDepReg's kill, it's safe to replace AntiDepReg with >> NewReg. >> + assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices >> [AntiDepReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for AntiDepReg!"); >> + assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == >> ~0u)) && >> + "Kill and Def maps aren't consistent for NewReg!"); >> + if (KillIndices[NewReg] != ~0u || >> + Classes[NewReg] == reinterpret_cast >> (-1) || >> + KillIndices[AntiDepReg] > DefIndices[NewReg]) >> + continue; >> + return NewReg; >> + } >> + >> + // No registers are free and available! >> + return 0; >> +} >> + >> +unsigned CriticalAntiDepBreaker:: >> +BreakAntiDependencies(std::vector& SUnits, >> + MachineBasicBlock::iterator& Begin, >> + MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex) { >> + // The code below assumes that there is at least one instruction, >> + // so just duck out immediately if the block is empty. >> + if (SUnits.empty()) return 0; >> + >> + // Find the node at the bottom of the critical path. >> + SUnit *Max = 0; >> + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >> + SUnit *SU = &SUnits[i]; >> + if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + >> Max->Latency) >> + Max = SU; >> + } >> + >> +#ifndef NDEBUG >> + { >> + DEBUG(errs() << "Critical path has total latency " >> + << (Max->getDepth() + Max->Latency) << "\n"); >> + DEBUG(errs() << "Available regs:"); >> + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >> + if (KillIndices[Reg] == ~0u) >> + DEBUG(errs() << " " << TRI->getName(Reg)); >> + } >> + DEBUG(errs() << '\n'); >> + } >> +#endif >> + >> + // Track progress along the critical path through the SUnit >> graph as we walk >> + // the instructions. >> + SUnit *CriticalPathSU = Max; >> + MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >> + >> + // Consider this pattern: >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // A = ... >> + // ... = A >> + // There are three anti-dependencies here, and without special >> care, >> + // we'd break all of them using the same register: >> + // A = ... >> + // ... = A >> + // B = ... >> + // ... = B >> + // B = ... >> + // ... = B >> + // B = ... >> + // ... = B >> + // because at each anti-dependence, B is the first register that >> + // isn't A which is free. This re-introduces anti-dependencies >> + // at all but one of the original anti-dependencies that we were >> + // trying to break. To avoid this, keep track of the most recent >> + // register that each register was replaced with, avoid >> + // using it to repair an anti-dependence on the same register. >> + // This lets us produce this: >> + // A = ... >> + // ... = A >> + // B = ... >> + // ... = B >> + // C = ... >> + // ... = C >> + // B = ... >> + // ... = B >> + // This still has an anti-dependence on B, but at least it isn't >> on the >> + // original critical path. >> + // >> + // TODO: If we tracked more than one register here, we could >> potentially >> + // fix that remaining critical edge too. This is a little more >> involved, >> + // because unlike the most recent register, less recent >> registers should >> + // still be considered, though only if no other registers are >> available. >> + unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = >> {}; >> + >> + // Attempt to break anti-dependence edges on the critical path. >> Walk the >> + // instructions from the bottom up, tracking information about >> liveness >> + // as we go to help determine which registers are available. >> + unsigned Broken = 0; >> + unsigned Count = InsertPosIndex - 1; >> + for (MachineBasicBlock::iterator I = End, E = Begin; >> + I != E; --Count) { >> + MachineInstr *MI = --I; >> + >> + // Check if this instruction has a dependence on the critical >> path that >> + // is an anti-dependence that we may be able to break. If it >> is, set >> + // AntiDepReg to the non-zero register associated with the >> anti-dependence. >> + // >> + // We limit our attention to the critical path as a heuristic >> to avoid >> + // breaking anti-dependence edges that aren't going to >> significantly >> + // impact the overall schedule. There are a limited number of >> registers >> + // and we want to save them for the important edges. >> + // >> + // TODO: Instructions with multiple defs could have multiple >> + // anti-dependencies. The current code here only knows how to >> break one >> + // edge per instruction. Note that we'd have to be able to >> break all of >> + // the anti-dependencies in an instruction in order to be >> effective. >> + unsigned AntiDepReg = 0; >> + if (MI == CriticalPathMI) { >> + if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >> + SUnit *NextSU = Edge->getSUnit(); >> + >> + // Only consider anti-dependence edges. >> + if (Edge->getKind() == SDep::Anti) { >> + AntiDepReg = Edge->getReg(); >> + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >> + if (!AllocatableSet.test(AntiDepReg)) >> + // Don't break anti-dependencies on non-allocatable >> registers. >> + AntiDepReg = 0; >> + else if (KeepRegs.count(AntiDepReg)) >> + // Don't break anti-dependencies if an use down below >> requires >> + // this exact register. >> + AntiDepReg = 0; >> + else { >> + // If the SUnit has other dependencies on the SUnit >> that it >> + // anti-depends on, don't bother breaking the anti- >> dependency >> + // since those edges would prevent such units from being >> + // scheduled past each other regardless. >> + // >> + // Also, if there are dependencies on other SUnits >> with the >> + // same register as the anti-dependency, don't attempt >> to >> + // break it. >> + for (SUnit::pred_iterator P = CriticalPathSU- >> >Preds.begin(), >> + PE = CriticalPathSU->Preds.end(); P != PE; ++P) >> + if (P->getSUnit() == NextSU ? >> + (P->getKind() != SDep::Anti || P->getReg() != >> AntiDepReg) : >> + (P->getKind() == SDep::Data && P->getReg() == >> AntiDepReg)) { >> + AntiDepReg = 0; >> + break; >> + } >> + } >> + } >> + CriticalPathSU = NextSU; >> + CriticalPathMI = CriticalPathSU->getInstr(); >> + } else { >> + // We've reached the end of the critical path. >> + CriticalPathSU = 0; >> + CriticalPathMI = 0; >> + } >> + } >> + >> + PrescanInstruction(MI); >> + >> + if (MI->getDesc().hasExtraDefRegAllocReq()) >> + // If this instruction's defs have special allocation >> requirement, don't >> + // break this anti-dependency. >> + AntiDepReg = 0; >> + else if (AntiDepReg) { >> + // If this instruction has a use of AntiDepReg, breaking it >> + // is invalid. >> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> + MachineOperand &MO = MI->getOperand(i); >> + if (!MO.isReg()) continue; >> + unsigned Reg = MO.getReg(); >> + if (Reg == 0) continue; >> + if (MO.isUse() && AntiDepReg == Reg) { >> + AntiDepReg = 0; >> + break; >> + } >> + } >> + } >> + >> + // Determine AntiDepReg's register class, if it is live and is >> + // consistently used within a single class. >> + const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes >> [AntiDepReg] : 0; >> + assert((AntiDepReg == 0 || RC != NULL) && >> + "Register should be live if it's causing an anti- >> dependence!"); >> + if (RC == reinterpret_cast(-1)) >> + AntiDepReg = 0; >> + >> + // Look for a suitable register to use to break the anti- >> depenence. >> + // >> + // TODO: Instead of picking the first free register, consider >> which might >> + // be the best. >> + if (AntiDepReg != 0) { >> + if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >> + LastNewReg >> [AntiDepReg], >> + RC)) { >> + DEBUG(errs() << "Breaking anti-dependence edge on " >> + << TRI->getName(AntiDepReg) >> + << " with " << RegRefs.count(AntiDepReg) << " >> references" >> + << " using " << TRI->getName(NewReg) << "!\n"); >> + >> + // Update the references to the old register to refer to >> the new >> + // register. >> + std::pair> *>::iterator, >> + std::multimap> *>::iterator> >> + Range = RegRefs.equal_range(AntiDepReg); >> + for (std::multimap::iterator >> + Q = Range.first, QE = Range.second; Q != QE; ++Q) >> + Q->second->setReg(NewReg); >> + >> + // We just went back in time and modified history; the >> + // liveness information for the anti-depenence reg is now >> + // inconsistent. Set the state as if it were dead. >> + Classes[NewReg] = Classes[AntiDepReg]; >> + DefIndices[NewReg] = DefIndices[AntiDepReg]; >> + KillIndices[NewReg] = KillIndices[AntiDepReg]; >> + assert(((KillIndices[NewReg] == ~0u) != >> + (DefIndices[NewReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for NewReg!"); >> + >> + Classes[AntiDepReg] = 0; >> + DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >> + KillIndices[AntiDepReg] = ~0u; >> + assert(((KillIndices[AntiDepReg] == ~0u) != >> + (DefIndices[AntiDepReg] == ~0u)) && >> + "Kill and Def maps aren't consistent for AntiDepReg!"); >> + >> + RegRefs.erase(AntiDepReg); >> + LastNewReg[AntiDepReg] = NewReg; >> + ++Broken; >> + } >> + } >> + >> + ScanInstruction(MI, Count); >> + } >> + >> + return Broken; >> +} >> >> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85127&view=auto >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (added) >> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 >> 11:59:04 2009 >> @@ -0,0 +1,95 @@ >> +//=- llvm/CodeGen/CriticalAntiDepBreaker.h - Anti-Dep Support -*- C >> ++ -*-=// >> +// >> +// The LLVM Compiler Infrastructure >> +// >> +// This file is distributed under the University of Illinois Open >> Source >> +// License. See LICENSE.TXT for details. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> +// >> +// This file implements the CriticalAntiDepBreaker class, which >> +// implements register anti-dependence breaking along a blocks >> +// critical path during post-RA scheduler. >> +// >> +// >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> + >> +#ifndef LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >> +#define LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >> + >> +#include "llvm/CodeGen/AntiDepBreaker.h" >> +#include "llvm/CodeGen/MachineBasicBlock.h" >> +#include "llvm/CodeGen/MachineFrameInfo.h" >> +#include "llvm/CodeGen/MachineFunction.h" >> +#include "llvm/CodeGen/MachineRegisterInfo.h" >> +#include "llvm/CodeGen/ScheduleDAG.h" >> +#include "llvm/Target/TargetRegisterInfo.h" >> +#include "llvm/ADT/BitVector.h" >> +#include "llvm/ADT/SmallSet.h" >> + >> +namespace llvm { >> + class CriticalAntiDepBreaker : public AntiDepBreaker { >> + MachineFunction& MF; >> + MachineRegisterInfo &MRI; >> + const TargetRegisterInfo *TRI; >> + >> + /// AllocatableSet - The set of allocatable registers. >> + /// We'll be ignoring anti-dependencies on non-allocatable >> registers, >> + /// because they may not be safe to break. >> + const BitVector AllocatableSet; >> + >> + /// Classes - For live regs that are only used in one register >> class in a >> + /// live range, the register class. If the register is not >> live, the >> + /// corresponding value is null. If the register is live but >> used in >> + /// multiple register classes, the corresponding value is -1 >> casted to a >> + /// pointer. >> + const TargetRegisterClass * >> + Classes[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// RegRegs - Map registers to all their references within a >> live range. >> + std::multimap RegRefs; >> + >> + /// KillIndices - The index of the most recent kill (proceding >> bottom-up), >> + /// or ~0u if the register is not live. >> + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// DefIndices - The index of the most recent complete def >> (proceding bottom >> + /// up), or ~0u if the register is live. >> + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >> + >> + /// KeepRegs - A set of registers which are live and cannot be >> changed to >> + /// break anti-dependencies. >> + SmallSet KeepRegs; >> + >> + public: >> + CriticalAntiDepBreaker(MachineFunction& MFi); >> + ~CriticalAntiDepBreaker(); >> + >> + /// Start - Initialize anti-dep breaking for a new basic block. >> + void StartBlock(MachineBasicBlock *BB); >> + >> + /// BreakAntiDependencies - Identifiy anti-dependencies along >> the critical path >> + /// of the ScheduleDAG and break them by renaming registers. >> + /// >> + unsigned BreakAntiDependencies(std::vector& SUnits, >> + MachineBasicBlock::iterator& >> Begin, >> + MachineBasicBlock::iterator& End, >> + unsigned InsertPosIndex); >> + >> + /// Observe - Update liveness information to account for the >> current >> + /// instruction, which will not be scheduled. >> + /// >> + void Observe(MachineInstr *MI, unsigned Count, unsigned >> InsertPosIndex); >> + >> + /// Finish - Finish anti-dep breaking for a basic block. >> + void FinishBlock(); >> + >> + private: >> + void PrescanInstruction(MachineInstr *MI); >> + void ScanInstruction(MachineInstr *MI, unsigned Count); >> + unsigned findSuitableFreeRegister(unsigned AntiDepReg, >> + unsigned LastNewReg, >> + const TargetRegisterClass *); >> + }; >> +} >> + >> +#endif >> >> Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) >> +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 >> 11:59:04 2009 >> @@ -19,6 +19,7 @@ >> // >> = >> = >> = >> ----------------------------------------------------------------------= >> ==// >> >> #define DEBUG_TYPE "post-RA-sched" >> +#include "CriticalAntiDepBreaker.h" >> #include "ExactHazardRecognizer.h" >> #include "SimpleHazardRecognizer.h" >> #include "ScheduleDAGInstrs.h" >> @@ -40,6 +41,7 @@ >> #include "llvm/Support/Debug.h" >> #include "llvm/Support/ErrorHandling.h" >> #include "llvm/Support/raw_ostream.h" >> +#include "llvm/ADT/BitVector.h" >> #include "llvm/ADT/Statistic.h" >> #include >> #include >> @@ -47,6 +49,7 @@ >> >> STATISTIC(NumNoops, "Number of noops inserted"); >> STATISTIC(NumStalls, "Number of pipeline stalls"); >> +STATISTIC(NumFixedAnti, "Number of fixed anti-dependencies"); >> >> // Post-RA scheduling is enabled with >> // TargetSubtarget.enablePostRAScheduler(). This flag can be used to >> @@ -55,10 +58,11 @@ >> EnablePostRAScheduler("post-RA-scheduler", >> cl::desc("Enable scheduling after register >> allocation"), >> cl::init(false), cl::Hidden); >> -static cl::opt >> +static cl::opt >> EnableAntiDepBreaking("break-anti-dependencies", >> - cl::desc("Break post-RA scheduling anti- >> dependencies"), >> - cl::init(true), cl::Hidden); >> + cl::desc("Break post-RA scheduling anti- >> dependencies: " >> + "\"critical\", \"all\", or \"none >> \""), >> + cl::init("none"), cl::Hidden); >> static cl::opt >> EnablePostRAHazardAvoidance("avoid-hazards", >> cl::desc("Enable exact hazard avoidance"), >> @@ -116,56 +120,30 @@ >> /// Topo - A topological ordering for SUnits. >> ScheduleDAGTopologicalSort Topo; >> >> - /// AllocatableSet - The set of allocatable registers. >> - /// We'll be ignoring anti-dependencies on non-allocatable >> registers, >> - /// because they may not be safe to break. >> - const BitVector AllocatableSet; >> - >> /// HazardRec - The hazard recognizer to use. >> ScheduleHazardRecognizer *HazardRec; >> >> + /// AntiDepBreak - Anti-dependence breaking object, or NULL if >> none >> + AntiDepBreaker *AntiDepBreak; >> + >> /// AA - AliasAnalysis for making memory reference queries. >> AliasAnalysis *AA; >> >> - /// AntiDepMode - Anti-dependence breaking mode >> - TargetSubtarget::AntiDepBreakMode AntiDepMode; >> - >> - /// Classes - For live regs that are only used in one register >> class in a >> - /// live range, the register class. If the register is not >> live, the >> - /// corresponding value is null. If the register is live but >> used in >> - /// multiple register classes, the corresponding value is -1 >> casted to a >> - /// pointer. >> - const TargetRegisterClass * >> - Classes[TargetRegisterInfo::FirstVirtualRegister]; >> - >> - /// RegRegs - Map registers to all their references within a >> live range. >> - std::multimap RegRefs; >> - >> /// KillIndices - The index of the most recent kill (proceding >> bottom-up), >> /// or ~0u if the register is not live. >> unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >> >> - /// DefIndices - The index of the most recent complete def >> (proceding bottom >> - /// up), or ~0u if the register is live. >> - unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >> - >> - /// KeepRegs - A set of registers which are live and cannot be >> changed to >> - /// break anti-dependencies. >> - SmallSet KeepRegs; >> - >> public: >> SchedulePostRATDList(MachineFunction &MF, >> const MachineLoopInfo &MLI, >> const MachineDominatorTree &MDT, >> ScheduleHazardRecognizer *HR, >> - AliasAnalysis *aa, >> - TargetSubtarget::AntiDepBreakMode adm) >> + AntiDepBreaker *ADB, >> + AliasAnalysis *aa) >> : ScheduleDAGInstrs(MF, MLI, MDT), Topo(SUnits), >> - AllocatableSet(TRI->getAllocatableSet(MF)), >> - HazardRec(HR), AA(aa), AntiDepMode(adm) {} >> + HazardRec(HR), AntiDepBreak(ADB), AA(aa) {} >> >> ~SchedulePostRATDList() { >> - delete HazardRec; >> } >> >> /// StartBlock - Initialize register live-range state for >> scheduling in >> @@ -177,11 +155,6 @@ >> /// >> void Schedule(); >> >> - /// FixupKills - Fix register kill flags that have been made >> - /// invalid due to scheduling >> - /// >> - void FixupKills(MachineBasicBlock *MBB); >> - >> /// Observe - Update liveness information to account for the >> current >> /// instruction, which will not be scheduled. >> /// >> @@ -191,17 +164,16 @@ >> /// >> void FinishBlock(); >> >> + /// FixupKills - Fix register kill flags that have been made >> + /// invalid due to scheduling >> + /// >> + void FixupKills(MachineBasicBlock *MBB); >> + >> private: >> - void PrescanInstruction(MachineInstr *MI); >> - void ScanInstruction(MachineInstr *MI, unsigned Count); >> void ReleaseSucc(SUnit *SU, SDep *SuccEdge); >> void ReleaseSuccessors(SUnit *SU); >> void ScheduleNodeTopDown(SUnit *SU, unsigned CurCycle); >> void ListScheduleTopDown(); >> - bool BreakAntiDependencies(); >> - unsigned findSuitableFreeRegister(unsigned AntiDepReg, >> - unsigned LastNewReg, >> - const TargetRegisterClass *); >> void StartBlockForKills(MachineBasicBlock *BB); >> >> // ToggleKillFlag - Toggle a register operand kill flag. Other >> @@ -250,8 +222,9 @@ >> >> // Check for antidep breaking override... >> if (EnableAntiDepBreaking.getPosition() > 0) { >> - AntiDepMode = (EnableAntiDepBreaking) ? >> - TargetSubtarget::ANTIDEP_CRITICAL : >> TargetSubtarget::ANTIDEP_NONE; >> + AntiDepMode = (EnableAntiDepBreaking == "all") ? >> TargetSubtarget::ANTIDEP_ALL : >> + (EnableAntiDepBreaking == "critical") ? >> TargetSubtarget::ANTIDEP_CRITICAL : >> + TargetSubtarget::ANTIDEP_NONE; >> } >> >> DEBUG(errs() << "PostRAScheduler\n"); >> @@ -262,8 +235,12 @@ >> ScheduleHazardRecognizer *HR = EnablePostRAHazardAvoidance ? >> (ScheduleHazardRecognizer *)new ExactHazardRecognizer >> (InstrItins) : >> (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); >> + AntiDepBreaker *ADB = >> + (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME >> */ : >> + (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? >> + new CriticalAntiDepBreaker(Fn) : NULL; >> >> - SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, AA, AntiDepMode); >> + SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); >> >> // Loop over all of the basic blocks >> for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end(); >> @@ -311,6 +288,9 @@ >> Scheduler.FixupKills(MBB); >> } >> >> + delete HR; >> + delete ADB; >> + >> return true; >> } >> >> @@ -321,78 +301,10 @@ >> // Call the superclass. >> ScheduleDAGInstrs::StartBlock(BB); >> >> - // Reset the hazard recognizer. >> + // Reset the hazard recognizer and anti-dep breaker. >> HazardRec->Reset(); >> - >> - // Clear out the register class data. >> - std::fill(Classes, array_endof(Classes), >> - static_cast(0)); >> - >> - // Initialize the indices to indicate that no registers are live. >> - std::fill(KillIndices, array_endof(KillIndices), ~0u); >> - std::fill(DefIndices, array_endof(DefIndices), BB->size()); >> - >> - // Clear "do not change" set. >> - KeepRegs.clear(); >> - >> - bool IsReturnBlock = (!BB->empty() && BB->back().getDesc >> ().isReturn()); >> - >> - // Determine the live-out physregs for this block. >> - if (IsReturnBlock) { >> - // In a return block, examine the function live-out regs. >> - for (MachineRegisterInfo::liveout_iterator I = >> MRI.liveout_begin(), >> - E = MRI.liveout_end(); I != E; ++I) { >> - unsigned Reg = *I; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } else { >> - // In a non-return block, examine the live-in regs of all >> successors. >> - for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >> - SE = BB->succ_end(); SI != SE; ++SI) >> - for (MachineBasicBlock::livein_iterator I = (*SI)- >> >livein_begin(), >> - E = (*SI)->livein_end(); I != E; ++I) { >> - unsigned Reg = *I; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); >> *Alias; ++Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast> *>(-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } >> - >> - // Mark live-out callee-saved registers. In a return block this is >> - // all callee-saved registers. In non-return this is any >> - // callee-saved register that is not saved in the prolog. >> - const MachineFrameInfo *MFI = MF.getFrameInfo(); >> - BitVector Pristine = MFI->getPristineRegs(BB); >> - for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >> - unsigned Reg = *I; >> - if (!IsReturnBlock && !Pristine.test(Reg)) continue; >> - Classes[Reg] = reinterpret_cast(-1); >> - KillIndices[Reg] = BB->size(); >> - DefIndices[Reg] = ~0u; >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - KillIndices[AliasReg] = BB->size(); >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->StartBlock(BB); >> } >> >> /// Schedule - Schedule the instruction range using list scheduling. >> @@ -403,8 +315,11 @@ >> // Build the scheduling graph. >> BuildSchedGraph(AA); >> >> - if (AntiDepMode != TargetSubtarget::ANTIDEP_NONE) { >> - if (BreakAntiDependencies()) { >> + if (AntiDepBreak != NULL) { >> + unsigned Broken = >> + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, >> + InsertPosIndex); >> + if (Broken > 0) { >> // We made changes. Update the dependency graph. >> // Theoretically we could update the graph in place: >> // When a live range is changed to use a different register, >> remove >> @@ -415,6 +330,8 @@ >> EntrySU = SUnit(); >> ExitSU = SUnit(); >> BuildSchedGraph(AA); >> + >> + NumFixedAnti += Broken; >> } >> } >> >> @@ -432,436 +349,20 @@ >> /// instruction, which will not be scheduled. >> /// >> void SchedulePostRATDList::Observe(MachineInstr *MI, unsigned >> Count) { >> - assert(Count < InsertPosIndex && "Instruction index out of >> expected range!"); >> - >> - // Any register which was defined within the previous scheduling >> region >> - // may have been rescheduled and its lifetime may overlap with >> registers >> - // in ways not reflected in our current liveness state. For each >> such >> - // register, adjust the liveness state to be conservatively >> correct. >> - for (unsigned Reg = 0; Reg != >> TargetRegisterInfo::FirstVirtualRegister; ++Reg) >> - if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= >> Count) { >> - assert(KillIndices[Reg] == ~0u && "Clobbered register is >> live!"); >> - // Mark this register to be non-renamable. >> - Classes[Reg] = reinterpret_cast(-1); >> - // Move the def index to the end of the previous region, to >> reflect >> - // that the def could theoretically have been scheduled at >> the end. >> - DefIndices[Reg] = InsertPosIndex; >> - } >> - >> - PrescanInstruction(MI); >> - ScanInstruction(MI, Count); >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->Observe(MI, Count, InsertPosIndex); >> } >> >> /// FinishBlock - Clean up register live-range state. >> /// >> void SchedulePostRATDList::FinishBlock() { >> - RegRefs.clear(); >> + if (AntiDepBreak != NULL) >> + AntiDepBreak->FinishBlock(); >> >> // Call the superclass. >> ScheduleDAGInstrs::FinishBlock(); >> } >> >> -/// CriticalPathStep - Return the next SUnit after SU on the >> bottom-up >> -/// critical path. >> -static SDep *CriticalPathStep(SUnit *SU) { >> - SDep *Next = 0; >> - unsigned NextDepth = 0; >> - // Find the predecessor edge with the greatest depth. >> - for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU- >> >Preds.end(); >> - P != PE; ++P) { >> - SUnit *PredSU = P->getSUnit(); >> - unsigned PredLatency = P->getLatency(); >> - unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >> - // In the case of a latency tie, prefer an anti-dependency >> edge over >> - // other types of edges. >> - if (NextDepth < PredTotalLatency || >> - (NextDepth == PredTotalLatency && P->getKind() == >> SDep::Anti)) { >> - NextDepth = PredTotalLatency; >> - Next = &*P; >> - } >> - } >> - return Next; >> -} >> - >> -void SchedulePostRATDList::PrescanInstruction(MachineInstr *MI) { >> - // Scan the register operands for this instruction and update >> - // Classes and RegRefs. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - const TargetRegisterClass *NewRC = 0; >> - >> - if (i < MI->getDesc().getNumOperands()) >> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> - >> - // For now, only allow the register to be changed if its >> register >> - // class is consistent across all uses. >> - if (!Classes[Reg] && NewRC) >> - Classes[Reg] = NewRC; >> - else if (!NewRC || Classes[Reg] != NewRC) >> - Classes[Reg] = reinterpret_cast(-1); >> - >> - // Now check for aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - // If an alias of the reg is used during the live range, >> give up. >> - // Note that this allows us to skip checking if AntiDepReg >> - // overlaps with any of the aliases, among other things. >> - unsigned AliasReg = *Alias; >> - if (Classes[AliasReg]) { >> - Classes[AliasReg] = reinterpret_cast >> (-1); >> - Classes[Reg] = reinterpret_cast(-1); >> - } >> - } >> - >> - // If we're still willing to consider this register, note the >> reference. >> - if (Classes[Reg] != reinterpret_cast(-1)) >> - RegRefs.insert(std::make_pair(Reg, &MO)); >> - >> - // It's not safe to change register allocation for source >> operands of >> - // that have special allocation requirements. >> - if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >> - if (KeepRegs.insert(Reg)) { >> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> - *Subreg; ++Subreg) >> - KeepRegs.insert(*Subreg); >> - } >> - } >> - } >> -} >> - >> -void SchedulePostRATDList::ScanInstruction(MachineInstr *MI, >> - unsigned Count) { >> - // Update liveness. >> - // Proceding upwards, registers that are defed but not used in >> this >> - // instruction are now dead. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (!MO.isDef()) continue; >> - // Ignore two-addr defs. >> - if (MI->isRegTiedToUseOperand(i)) continue; >> - >> - DefIndices[Reg] = Count; >> - KillIndices[Reg] = ~0u; >> - assert(((KillIndices[Reg] == ~0u) != >> - (DefIndices[Reg] == ~0u)) && >> - "Kill and Def maps aren't consistent for Reg!"); >> - KeepRegs.erase(Reg); >> - Classes[Reg] = 0; >> - RegRefs.erase(Reg); >> - // Repeat, for all subregs. >> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >> - *Subreg; ++Subreg) { >> - unsigned SubregReg = *Subreg; >> - DefIndices[SubregReg] = Count; >> - KillIndices[SubregReg] = ~0u; >> - KeepRegs.erase(SubregReg); >> - Classes[SubregReg] = 0; >> - RegRefs.erase(SubregReg); >> - } >> - // Conservatively mark super-registers as unusable. >> - for (const unsigned *Super = TRI->getSuperRegisters(Reg); >> - *Super; ++Super) { >> - unsigned SuperReg = *Super; >> - Classes[SuperReg] = reinterpret_cast >> (-1); >> - } >> - } >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (!MO.isUse()) continue; >> - >> - const TargetRegisterClass *NewRC = 0; >> - if (i < MI->getDesc().getNumOperands()) >> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >> - >> - // For now, only allow the register to be changed if its >> register >> - // class is consistent across all uses. >> - if (!Classes[Reg] && NewRC) >> - Classes[Reg] = NewRC; >> - else if (!NewRC || Classes[Reg] != NewRC) >> - Classes[Reg] = reinterpret_cast(-1); >> - >> - RegRefs.insert(std::make_pair(Reg, &MO)); >> - >> - // It wasn't previously live but now it is, this is a kill. >> - if (KillIndices[Reg] == ~0u) { >> - KillIndices[Reg] = Count; >> - DefIndices[Reg] = ~0u; >> - assert(((KillIndices[Reg] == ~0u) != >> - (DefIndices[Reg] == ~0u)) && >> - "Kill and Def maps aren't consistent for Reg!"); >> - } >> - // Repeat, for all aliases. >> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; + >> +Alias) { >> - unsigned AliasReg = *Alias; >> - if (KillIndices[AliasReg] == ~0u) { >> - KillIndices[AliasReg] = Count; >> - DefIndices[AliasReg] = ~0u; >> - } >> - } >> - } >> -} >> - >> -unsigned >> -SchedulePostRATDList::findSuitableFreeRegister(unsigned AntiDepReg, >> - unsigned LastNewReg, >> - const >> TargetRegisterClass *RC) { >> - for (TargetRegisterClass::iterator R = RC->allocation_order_begin >> (MF), >> - RE = RC->allocation_order_end(MF); R != RE; ++R) { >> - unsigned NewReg = *R; >> - // Don't replace a register with itself. >> - if (NewReg == AntiDepReg) continue; >> - // Don't replace a register with one that was recently used to >> repair >> - // an anti-dependence with this AntiDepReg, because that would >> - // re-introduce that anti-dependence. >> - if (NewReg == LastNewReg) continue; >> - // If NewReg is dead and NewReg's most recent def is not before >> - // AntiDepReg's kill, it's safe to replace AntiDepReg with >> NewReg. >> - assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices >> [AntiDepReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for AntiDepReg!"); >> - assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == >> ~0u)) && >> - "Kill and Def maps aren't consistent for NewReg!"); >> - if (KillIndices[NewReg] != ~0u || >> - Classes[NewReg] == reinterpret_cast >> (-1) || >> - KillIndices[AntiDepReg] > DefIndices[NewReg]) >> - continue; >> - return NewReg; >> - } >> - >> - // No registers are free and available! >> - return 0; >> -} >> - >> -/// BreakAntiDependencies - Identifiy anti-dependencies along the >> critical path >> -/// of the ScheduleDAG and break them by renaming registers. >> -/// >> -bool SchedulePostRATDList::BreakAntiDependencies() { >> - // The code below assumes that there is at least one instruction, >> - // so just duck out immediately if the block is empty. >> - if (SUnits.empty()) return false; >> - >> - // Find the node at the bottom of the critical path. >> - SUnit *Max = 0; >> - for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >> - SUnit *SU = &SUnits[i]; >> - if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + >> Max->Latency) >> - Max = SU; >> - } >> - >> -#ifndef NDEBUG >> - { >> - DEBUG(errs() << "Critical path has total latency " >> - << (Max->getDepth() + Max->Latency) << "\n"); >> - DEBUG(errs() << "Available regs:"); >> - for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >> - if (KillIndices[Reg] == ~0u) >> - DEBUG(errs() << " " << TRI->getName(Reg)); >> - } >> - DEBUG(errs() << '\n'); >> - } >> -#endif >> - >> - // Track progress along the critical path through the SUnit >> graph as we walk >> - // the instructions. >> - SUnit *CriticalPathSU = Max; >> - MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >> - >> - // Consider this pattern: >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // A = ... >> - // ... = A >> - // There are three anti-dependencies here, and without special >> care, >> - // we'd break all of them using the same register: >> - // A = ... >> - // ... = A >> - // B = ... >> - // ... = B >> - // B = ... >> - // ... = B >> - // B = ... >> - // ... = B >> - // because at each anti-dependence, B is the first register that >> - // isn't A which is free. This re-introduces anti-dependencies >> - // at all but one of the original anti-dependencies that we were >> - // trying to break. To avoid this, keep track of the most recent >> - // register that each register was replaced with, avoid >> - // using it to repair an anti-dependence on the same register. >> - // This lets us produce this: >> - // A = ... >> - // ... = A >> - // B = ... >> - // ... = B >> - // C = ... >> - // ... = C >> - // B = ... >> - // ... = B >> - // This still has an anti-dependence on B, but at least it isn't >> on the >> - // original critical path. >> - // >> - // TODO: If we tracked more than one register here, we could >> potentially >> - // fix that remaining critical edge too. This is a little more >> involved, >> - // because unlike the most recent register, less recent >> registers should >> - // still be considered, though only if no other registers are >> available. >> - unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = >> {}; >> - >> - // Attempt to break anti-dependence edges on the critical path. >> Walk the >> - // instructions from the bottom up, tracking information about >> liveness >> - // as we go to help determine which registers are available. >> - bool Changed = false; >> - unsigned Count = InsertPosIndex - 1; >> - for (MachineBasicBlock::iterator I = InsertPos, E = Begin; >> - I != E; --Count) { >> - MachineInstr *MI = --I; >> - >> - // Check if this instruction has a dependence on the critical >> path that >> - // is an anti-dependence that we may be able to break. If it >> is, set >> - // AntiDepReg to the non-zero register associated with the >> anti-dependence. >> - // >> - // We limit our attention to the critical path as a heuristic >> to avoid >> - // breaking anti-dependence edges that aren't going to >> significantly >> - // impact the overall schedule. There are a limited number of >> registers >> - // and we want to save them for the important edges. >> - // >> - // TODO: Instructions with multiple defs could have multiple >> - // anti-dependencies. The current code here only knows how to >> break one >> - // edge per instruction. Note that we'd have to be able to >> break all of >> - // the anti-dependencies in an instruction in order to be >> effective. >> - unsigned AntiDepReg = 0; >> - if (MI == CriticalPathMI) { >> - if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >> - SUnit *NextSU = Edge->getSUnit(); >> - >> - // Only consider anti-dependence edges. >> - if (Edge->getKind() == SDep::Anti) { >> - AntiDepReg = Edge->getReg(); >> - assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >> - if (!AllocatableSet.test(AntiDepReg)) >> - // Don't break anti-dependencies on non-allocatable >> registers. >> - AntiDepReg = 0; >> - else if (KeepRegs.count(AntiDepReg)) >> - // Don't break anti-dependencies if an use down below >> requires >> - // this exact register. >> - AntiDepReg = 0; >> - else { >> - // If the SUnit has other dependencies on the SUnit >> that it >> - // anti-depends on, don't bother breaking the anti- >> dependency >> - // since those edges would prevent such units from being >> - // scheduled past each other regardless. >> - // >> - // Also, if there are dependencies on other SUnits >> with the >> - // same register as the anti-dependency, don't attempt >> to >> - // break it. >> - for (SUnit::pred_iterator P = CriticalPathSU- >> >Preds.begin(), >> - PE = CriticalPathSU->Preds.end(); P != PE; ++P) >> - if (P->getSUnit() == NextSU ? >> - (P->getKind() != SDep::Anti || P->getReg() != >> AntiDepReg) : >> - (P->getKind() == SDep::Data && P->getReg() == >> AntiDepReg)) { >> - AntiDepReg = 0; >> - break; >> - } >> - } >> - } >> - CriticalPathSU = NextSU; >> - CriticalPathMI = CriticalPathSU->getInstr(); >> - } else { >> - // We've reached the end of the critical path. >> - CriticalPathSU = 0; >> - CriticalPathMI = 0; >> - } >> - } >> - >> - PrescanInstruction(MI); >> - >> - if (MI->getDesc().hasExtraDefRegAllocReq()) >> - // If this instruction's defs have special allocation >> requirement, don't >> - // break this anti-dependency. >> - AntiDepReg = 0; >> - else if (AntiDepReg) { >> - // If this instruction has a use of AntiDepReg, breaking it >> - // is invalid. >> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >> - MachineOperand &MO = MI->getOperand(i); >> - if (!MO.isReg()) continue; >> - unsigned Reg = MO.getReg(); >> - if (Reg == 0) continue; >> - if (MO.isUse() && AntiDepReg == Reg) { >> - AntiDepReg = 0; >> - break; >> - } >> - } >> - } >> - >> - // Determine AntiDepReg's register class, if it is live and is >> - // consistently used within a single class. >> - const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes >> [AntiDepReg] : 0; >> - assert((AntiDepReg == 0 || RC != NULL) && >> - "Register should be live if it's causing an anti- >> dependence!"); >> - if (RC == reinterpret_cast(-1)) >> - AntiDepReg = 0; >> - >> - // Look for a suitable register to use to break the anti- >> depenence. >> - // >> - // TODO: Instead of picking the first free register, consider >> which might >> - // be the best. >> - if (AntiDepReg != 0) { >> - if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >> - LastNewReg >> [AntiDepReg], >> - RC)) { >> - DEBUG(errs() << "Breaking anti-dependence edge on " >> - << TRI->getName(AntiDepReg) >> - << " with " << RegRefs.count(AntiDepReg) << " >> references" >> - << " using " << TRI->getName(NewReg) << "!\n"); >> - >> - // Update the references to the old register to refer to >> the new >> - // register. >> - std::pair> *>::iterator, >> - std::multimap> *>::iterator> >> - Range = RegRefs.equal_range(AntiDepReg); >> - for (std::multimap::iterator >> - Q = Range.first, QE = Range.second; Q != QE; ++Q) >> - Q->second->setReg(NewReg); >> - >> - // We just went back in time and modified history; the >> - // liveness information for the anti-depenence reg is now >> - // inconsistent. Set the state as if it were dead. >> - Classes[NewReg] = Classes[AntiDepReg]; >> - DefIndices[NewReg] = DefIndices[AntiDepReg]; >> - KillIndices[NewReg] = KillIndices[AntiDepReg]; >> - assert(((KillIndices[NewReg] == ~0u) != >> - (DefIndices[NewReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for NewReg!"); >> - >> - Classes[AntiDepReg] = 0; >> - DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >> - KillIndices[AntiDepReg] = ~0u; >> - assert(((KillIndices[AntiDepReg] == ~0u) != >> - (DefIndices[AntiDepReg] == ~0u)) && >> - "Kill and Def maps aren't consistent for AntiDepReg!"); >> - >> - RegRefs.erase(AntiDepReg); >> - Changed = true; >> - LastNewReg[AntiDepReg] = NewReg; >> - } >> - } >> - >> - ScanInstruction(MI, Count); >> - } >> - >> - return Changed; >> -} >> - >> /// StartBlockForKills - Initialize register live-range state for >> updating kills >> /// >> void SchedulePostRATDList::StartBlockForKills(MachineBasicBlock >> *BB) { >> >> Modified: llvm/trunk/lib/Target/ARM/ARMSubtarget.h >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMSubtarget.h?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/lib/Target/ARM/ARMSubtarget.h (original) >> +++ llvm/trunk/lib/Target/ARM/ARMSubtarget.h Mon Oct 26 11:59:04 2009 >> @@ -130,7 +130,7 @@ >> /// for Thumb1. >> bool enablePostRAScheduler(CodeGenOpt::Level OptLevel, >> TargetSubtarget::AntiDepBreakMode& >> mode) const { >> - mode = TargetSubtarget::ANTIDEP_NONE; >> + mode = TargetSubtarget::ANTIDEP_CRITICAL; >> return PostRAScheduler && OptLevel >= CodeGenOpt::Default; >> } >> >> >> Modified: llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll?rev=85127&r1=85126&r2=85127&view=diff >> >> = >> = >> = >> = >> = >> = >> = >> = >> = >> ===================================================================== >> --- llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll (original) >> +++ llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Mon Oct >> 26 11:59:04 2009 >> @@ -1,7 +1,7 @@ >> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=false > %t >> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=none > %t >> ; RUN: grep {%xmm0} %t | count 14 >> ; RUN: not grep {%xmm1} %t >> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies > %t >> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti- >> dependencies=critical > %t >> ; RUN: grep {%xmm0} %t | count 7 >> ; RUN: grep {%xmm1} %t | count 7 >> >> >> >> _______________________________________________ >> llvm-commits mailing list >> llvm-commits at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits > From evan.cheng at apple.com Mon Oct 26 16:29:25 2009 From: evan.cheng at apple.com (Evan Cheng) Date: Mon, 26 Oct 2009 14:29:25 -0700 Subject: [llvm-commits] [llvm] r85127 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/CMakeLists.txt lib/CodeGen/CriticalAntiDepBreaker.cpp lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp lib/Target/ARM/ARMSubtarget.h test/CodeGen/X86/break-anti-dependencies.ll In-Reply-To: <35E02EF7-4CBB-4607-BC71-849E46D366D2@apple.com> References: <200910261659.n9QGx4Yj023936@zion.cs.uiuc.edu> <35E02EF7-4CBB-4607-BC71-849E46D366D2@apple.com> Message-ID: <62D81C23-FB80-4694-B347-5A7F94998189@apple.com> Targets must provide their own variants of the ScheduleHazardRecognizer. That's why it's exposed. Unless you feel there are reasons for targets to override the default AntiDepBreaker, it should be moved to lib/CodeGen. Thanks, Evan On Oct 26, 2009, at 2:27 PM, David Goodwin wrote: > I was just following the style of ScheduleHazardRecognizer, as it has the same "visibility" as AntiDepBreaker. But I can move it if that would be better... > > David > > On Oct 26, 2009, at 2:23 PM, Evan Cheng wrote: > >> It seems to be AntiDepBreaker.h should be inside lib/CodeGen. Is there a reason to expose it? >> >> Evan >> >> On Oct 26, 2009, at 9:59 AM, David Goodwin wrote: >> >>> Author: david_goodwin >>> Date: Mon Oct 26 11:59:04 2009 >>> New Revision: 85127 >>> >>> URL: http://llvm.org/viewvc/llvm-project?rev=85127&view=rev >>> Log: >>> Break anti-dependence breaking out into its own class. >>> >>> Added: >>> llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >>> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >>> llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >>> Modified: >>> llvm/trunk/lib/CodeGen/CMakeLists.txt >>> llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >>> llvm/trunk/lib/Target/ARM/ARMSubtarget.h >>> llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >>> >>> Added: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85127&view=auto >>> >>> ============================================================================== >>> --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (added) >>> +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 11:59:04 2009 >>> @@ -0,0 +1,56 @@ >>> +//=- llvm/CodeGen/AntiDepBreaker.h - Anti-Dependence Breaking -*- C++ -*-=// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===----------------------------------------------------------------------===// >>> +// >>> +// This file implements the AntiDepBreaker class, which implements >>> +// anti-dependence breaking heuristics for post-register-allocation scheduling. >>> +// >>> +//===----------------------------------------------------------------------===// >>> + >>> +#ifndef LLVM_CODEGEN_ANTIDEPBREAKER_H >>> +#define LLVM_CODEGEN_ANTIDEPBREAKER_H >>> + >>> +#include "llvm/CodeGen/MachineBasicBlock.h" >>> +#include "llvm/CodeGen/MachineFrameInfo.h" >>> +#include "llvm/CodeGen/MachineFunction.h" >>> +#include "llvm/CodeGen/MachineRegisterInfo.h" >>> +#include "llvm/CodeGen/ScheduleDAG.h" >>> +#include "llvm/Target/TargetRegisterInfo.h" >>> + >>> +namespace llvm { >>> + >>> +/// AntiDepBreaker - This class works into conjunction with the >>> +/// post-RA scheduler to rename registers to break register >>> +/// anti-dependencies. >>> +class AntiDepBreaker { >>> +public: >>> + /// Start - Initialize anti-dep breaking for a new basic block. >>> + virtual void StartBlock(MachineBasicBlock *BB) =0; >>> + >>> + /// BreakAntiDependencies - Identifiy anti-dependencies within a >>> + /// basic-block region and break them by renaming registers. Return >>> + /// the number of anti-dependencies broken. >>> + /// >>> + virtual unsigned BreakAntiDependencies(std::vector& SUnits, >>> + MachineBasicBlock::iterator& Begin, >>> + MachineBasicBlock::iterator& End, >>> + unsigned InsertPosIndex) =0; >>> + >>> + /// Observe - Update liveness information to account for the current >>> + /// instruction, which will not be scheduled. >>> + /// >>> + virtual void Observe(MachineInstr *MI, unsigned Count, >>> + unsigned InsertPosIndex) =0; >>> + >>> + /// Finish - Finish anti-dep breaking for a basic block. >>> + virtual void FinishBlock() =0; >>> +}; >>> + >>> +} >>> + >>> +#endif >>> >>> Modified: llvm/trunk/lib/CodeGen/CMakeLists.txt >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CMakeLists.txt?rev=85127&r1=85126&r2=85127&view=diff >>> >>> ============================================================================== >>> --- llvm/trunk/lib/CodeGen/CMakeLists.txt (original) >>> +++ llvm/trunk/lib/CodeGen/CMakeLists.txt Mon Oct 26 11:59:04 2009 >>> @@ -1,6 +1,7 @@ >>> add_llvm_library(LLVMCodeGen >>> BranchFolding.cpp >>> CodePlacementOpt.cpp >>> + CriticalAntiDepBreaker.cpp >>> DeadMachineInstructionElim.cpp >>> DwarfEHPrepare.cpp >>> ELFCodeEmitter.cpp >>> >>> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp?rev=85127&view=auto >>> >>> ============================================================================== >>> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp (added) >>> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.cpp Mon Oct 26 11:59:04 2009 >>> @@ -0,0 +1,539 @@ >>> +//===----- CriticalAntiDepBreaker.cpp - Anti-dep breaker -------- ---------===// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===----------------------------------------------------------------------===// >>> +// >>> +// This file implements the CriticalAntiDepBreaker class, which >>> +// implements register anti-dependence breaking along a blocks >>> +// critical path during post-RA scheduler. >>> +// >>> +//===----------------------------------------------------------------------===// >>> + >>> +#define DEBUG_TYPE "critical-antidep" >>> +#include "CriticalAntiDepBreaker.h" >>> +#include "llvm/CodeGen/MachineBasicBlock.h" >>> +#include "llvm/CodeGen/MachineFrameInfo.h" >>> +#include "llvm/Target/TargetMachine.h" >>> +#include "llvm/Target/TargetRegisterInfo.h" >>> +#include "llvm/Support/Debug.h" >>> +#include "llvm/Support/ErrorHandling.h" >>> +#include "llvm/Support/raw_ostream.h" >>> + >>> +using namespace llvm; >>> + >>> +CriticalAntiDepBreaker:: >>> +CriticalAntiDepBreaker(MachineFunction& MFi) : >>> + AntiDepBreaker(), MF(MFi), >>> + MRI(MF.getRegInfo()), >>> + TRI(MF.getTarget().getRegisterInfo()), >>> + AllocatableSet(TRI->getAllocatableSet(MF)) >>> +{ >>> +} >>> + >>> +CriticalAntiDepBreaker::~CriticalAntiDepBreaker() { >>> +} >>> + >>> +void CriticalAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { >>> + // Clear out the register class data. >>> + std::fill(Classes, array_endof(Classes), >>> + static_cast(0)); >>> + >>> + // Initialize the indices to indicate that no registers are live. >>> + std::fill(KillIndices, array_endof(KillIndices), ~0u); >>> + std::fill(DefIndices, array_endof(DefIndices), BB->size()); >>> + >>> + // Clear "do not change" set. >>> + KeepRegs.clear(); >>> + >>> + bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); >>> + >>> + // Determine the live-out physregs for this block. >>> + if (IsReturnBlock) { >>> + // In a return block, examine the function live-out regs. >>> + for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), >>> + E = MRI.liveout_end(); I != E; ++I) { >>> + unsigned Reg = *I; >>> + Classes[Reg] = reinterpret_cast(-1); >>> + KillIndices[Reg] = BB->size(); >>> + DefIndices[Reg] = ~0u; >>> + // Repeat, for all aliases. >>> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> + unsigned AliasReg = *Alias; >>> + Classes[AliasReg] = reinterpret_cast(-1); >>> + KillIndices[AliasReg] = BB->size(); >>> + DefIndices[AliasReg] = ~0u; >>> + } >>> + } >>> + } else { >>> + // In a non-return block, examine the live-in regs of all successors. >>> + for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >>> + SE = BB->succ_end(); SI != SE; ++SI) >>> + for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), >>> + E = (*SI)->livein_end(); I != E; ++I) { >>> + unsigned Reg = *I; >>> + Classes[Reg] = reinterpret_cast(-1); >>> + KillIndices[Reg] = BB->size(); >>> + DefIndices[Reg] = ~0u; >>> + // Repeat, for all aliases. >>> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> + unsigned AliasReg = *Alias; >>> + Classes[AliasReg] = reinterpret_cast(-1); >>> + KillIndices[AliasReg] = BB->size(); >>> + DefIndices[AliasReg] = ~0u; >>> + } >>> + } >>> + } >>> + >>> + // Mark live-out callee-saved registers. In a return block this is >>> + // all callee-saved registers. In non-return this is any >>> + // callee-saved register that is not saved in the prolog. >>> + const MachineFrameInfo *MFI = MF.getFrameInfo(); >>> + BitVector Pristine = MFI->getPristineRegs(BB); >>> + for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >>> + unsigned Reg = *I; >>> + if (!IsReturnBlock && !Pristine.test(Reg)) continue; >>> + Classes[Reg] = reinterpret_cast(-1); >>> + KillIndices[Reg] = BB->size(); >>> + DefIndices[Reg] = ~0u; >>> + // Repeat, for all aliases. >>> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> + unsigned AliasReg = *Alias; >>> + Classes[AliasReg] = reinterpret_cast(-1); >>> + KillIndices[AliasReg] = BB->size(); >>> + DefIndices[AliasReg] = ~0u; >>> + } >>> + } >>> +} >>> + >>> +void CriticalAntiDepBreaker::FinishBlock() { >>> + RegRefs.clear(); >>> + KeepRegs.clear(); >>> +} >>> + >>> +void CriticalAntiDepBreaker::Observe(MachineInstr *MI, unsigned Count, >>> + unsigned InsertPosIndex) { >>> + assert(Count < InsertPosIndex && "Instruction index out of expected range!"); >>> + >>> + // Any register which was defined within the previous scheduling region >>> + // may have been rescheduled and its lifetime may overlap with registers >>> + // in ways not reflected in our current liveness state. For each such >>> + // register, adjust the liveness state to be conservatively correct. >>> + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) >>> + if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { >>> + assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); >>> + // Mark this register to be non-renamable. >>> + Classes[Reg] = reinterpret_cast(-1); >>> + // Move the def index to the end of the previous region, to reflect >>> + // that the def could theoretically have been scheduled at the end. >>> + DefIndices[Reg] = InsertPosIndex; >>> + } >>> + >>> + PrescanInstruction(MI); >>> + ScanInstruction(MI, Count); >>> +} >>> + >>> +/// CriticalPathStep - Return the next SUnit after SU on the bottom-up >>> +/// critical path. >>> +static SDep *CriticalPathStep(SUnit *SU) { >>> + SDep *Next = 0; >>> + unsigned NextDepth = 0; >>> + // Find the predecessor edge with the greatest depth. >>> + for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); >>> + P != PE; ++P) { >>> + SUnit *PredSU = P->getSUnit(); >>> + unsigned PredLatency = P->getLatency(); >>> + unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >>> + // In the case of a latency tie, prefer an anti-dependency edge over >>> + // other types of edges. >>> + if (NextDepth < PredTotalLatency || >>> + (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { >>> + NextDepth = PredTotalLatency; >>> + Next = &*P; >>> + } >>> + } >>> + return Next; >>> +} >>> + >>> +void CriticalAntiDepBreaker::PrescanInstruction(MachineInstr *MI) { >>> + // Scan the register operands for this instruction and update >>> + // Classes and RegRefs. >>> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> + MachineOperand &MO = MI->getOperand(i); >>> + if (!MO.isReg()) continue; >>> + unsigned Reg = MO.getReg(); >>> + if (Reg == 0) continue; >>> + const TargetRegisterClass *NewRC = 0; >>> + >>> + if (i < MI->getDesc().getNumOperands()) >>> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >>> + >>> + // For now, only allow the register to be changed if its register >>> + // class is consistent across all uses. >>> + if (!Classes[Reg] && NewRC) >>> + Classes[Reg] = NewRC; >>> + else if (!NewRC || Classes[Reg] != NewRC) >>> + Classes[Reg] = reinterpret_cast(-1); >>> + >>> + // Now check for aliases. >>> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> + // If an alias of the reg is used during the live range, give up. >>> + // Note that this allows us to skip checking if AntiDepReg >>> + // overlaps with any of the aliases, among other things. >>> + unsigned AliasReg = *Alias; >>> + if (Classes[AliasReg]) { >>> + Classes[AliasReg] = reinterpret_cast(-1); >>> + Classes[Reg] = reinterpret_cast(-1); >>> + } >>> + } >>> + >>> + // If we're still willing to consider this register, note the reference. >>> + if (Classes[Reg] != reinterpret_cast(-1)) >>> + RegRefs.insert(std::make_pair(Reg, &MO)); >>> + >>> + // It's not safe to change register allocation for source operands of >>> + // that have special allocation requirements. >>> + if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >>> + if (KeepRegs.insert(Reg)) { >>> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >>> + *Subreg; ++Subreg) >>> + KeepRegs.insert(*Subreg); >>> + } >>> + } >>> + } >>> +} >>> + >>> +void CriticalAntiDepBreaker::ScanInstruction(MachineInstr *MI, >>> + unsigned Count) { >>> + // Update liveness. >>> + // Proceding upwards, registers that are defed but not used in this >>> + // instruction are now dead. >>> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> + MachineOperand &MO = MI->getOperand(i); >>> + if (!MO.isReg()) continue; >>> + unsigned Reg = MO.getReg(); >>> + if (Reg == 0) continue; >>> + if (!MO.isDef()) continue; >>> + // Ignore two-addr defs. >>> + if (MI->isRegTiedToUseOperand(i)) continue; >>> + >>> + DefIndices[Reg] = Count; >>> + KillIndices[Reg] = ~0u; >>> + assert(((KillIndices[Reg] == ~0u) != >>> + (DefIndices[Reg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for Reg!"); >>> + KeepRegs.erase(Reg); >>> + Classes[Reg] = 0; >>> + RegRefs.erase(Reg); >>> + // Repeat, for all subregs. >>> + for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >>> + *Subreg; ++Subreg) { >>> + unsigned SubregReg = *Subreg; >>> + DefIndices[SubregReg] = Count; >>> + KillIndices[SubregReg] = ~0u; >>> + KeepRegs.erase(SubregReg); >>> + Classes[SubregReg] = 0; >>> + RegRefs.erase(SubregReg); >>> + } >>> + // Conservatively mark super-registers as unusable. >>> + for (const unsigned *Super = TRI->getSuperRegisters(Reg); >>> + *Super; ++Super) { >>> + unsigned SuperReg = *Super; >>> + Classes[SuperReg] = reinterpret_cast(-1); >>> + } >>> + } >>> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> + MachineOperand &MO = MI->getOperand(i); >>> + if (!MO.isReg()) continue; >>> + unsigned Reg = MO.getReg(); >>> + if (Reg == 0) continue; >>> + if (!MO.isUse()) continue; >>> + >>> + const TargetRegisterClass *NewRC = 0; >>> + if (i < MI->getDesc().getNumOperands()) >>> + NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >>> + >>> + // For now, only allow the register to be changed if its register >>> + // class is consistent across all uses. >>> + if (!Classes[Reg] && NewRC) >>> + Classes[Reg] = NewRC; >>> + else if (!NewRC || Classes[Reg] != NewRC) >>> + Classes[Reg] = reinterpret_cast(-1); >>> + >>> + RegRefs.insert(std::make_pair(Reg, &MO)); >>> + >>> + // It wasn't previously live but now it is, this is a kill. >>> + if (KillIndices[Reg] == ~0u) { >>> + KillIndices[Reg] = Count; >>> + DefIndices[Reg] = ~0u; >>> + assert(((KillIndices[Reg] == ~0u) != >>> + (DefIndices[Reg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for Reg!"); >>> + } >>> + // Repeat, for all aliases. >>> + for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> + unsigned AliasReg = *Alias; >>> + if (KillIndices[AliasReg] == ~0u) { >>> + KillIndices[AliasReg] = Count; >>> + DefIndices[AliasReg] = ~0u; >>> + } >>> + } >>> + } >>> +} >>> + >>> +unsigned >>> +CriticalAntiDepBreaker::findSuitableFreeRegister(unsigned AntiDepReg, >>> + unsigned LastNewReg, >>> + const TargetRegisterClass *RC) { >>> + for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), >>> + RE = RC->allocation_order_end(MF); R != RE; ++R) { >>> + unsigned NewReg = *R; >>> + // Don't replace a register with itself. >>> + if (NewReg == AntiDepReg) continue; >>> + // Don't replace a register with one that was recently used to repair >>> + // an anti-dependence with this AntiDepReg, because that would >>> + // re-introduce that anti-dependence. >>> + if (NewReg == LastNewReg) continue; >>> + // If NewReg is dead and NewReg's most recent def is not before >>> + // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. >>> + assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for AntiDepReg!"); >>> + assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for NewReg!"); >>> + if (KillIndices[NewReg] != ~0u || >>> + Classes[NewReg] == reinterpret_cast(-1) || >>> + KillIndices[AntiDepReg] > DefIndices[NewReg]) >>> + continue; >>> + return NewReg; >>> + } >>> + >>> + // No registers are free and available! >>> + return 0; >>> +} >>> + >>> +unsigned CriticalAntiDepBreaker:: >>> +BreakAntiDependencies(std::vector& SUnits, >>> + MachineBasicBlock::iterator& Begin, >>> + MachineBasicBlock::iterator& End, >>> + unsigned InsertPosIndex) { >>> + // The code below assumes that there is at least one instruction, >>> + // so just duck out immediately if the block is empty. >>> + if (SUnits.empty()) return 0; >>> + >>> + // Find the node at the bottom of the critical path. >>> + SUnit *Max = 0; >>> + for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >>> + SUnit *SU = &SUnits[i]; >>> + if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) >>> + Max = SU; >>> + } >>> + >>> +#ifndef NDEBUG >>> + { >>> + DEBUG(errs() << "Critical path has total latency " >>> + << (Max->getDepth() + Max->Latency) << "\n"); >>> + DEBUG(errs() << "Available regs:"); >>> + for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >>> + if (KillIndices[Reg] == ~0u) >>> + DEBUG(errs() << " " << TRI->getName(Reg)); >>> + } >>> + DEBUG(errs() << '\n'); >>> + } >>> +#endif >>> + >>> + // Track progress along the critical path through the SUnit graph as we walk >>> + // the instructions. >>> + SUnit *CriticalPathSU = Max; >>> + MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >>> + >>> + // Consider this pattern: >>> + // A = ... >>> + // ... = A >>> + // A = ... >>> + // ... = A >>> + // A = ... >>> + // ... = A >>> + // A = ... >>> + // ... = A >>> + // There are three anti-dependencies here, and without special care, >>> + // we'd break all of them using the same register: >>> + // A = ... >>> + // ... = A >>> + // B = ... >>> + // ... = B >>> + // B = ... >>> + // ... = B >>> + // B = ... >>> + // ... = B >>> + // because at each anti-dependence, B is the first register that >>> + // isn't A which is free. This re-introduces anti-dependencies >>> + // at all but one of the original anti-dependencies that we were >>> + // trying to break. To avoid this, keep track of the most recent >>> + // register that each register was replaced with, avoid >>> + // using it to repair an anti-dependence on the same register. >>> + // This lets us produce this: >>> + // A = ... >>> + // ... = A >>> + // B = ... >>> + // ... = B >>> + // C = ... >>> + // ... = C >>> + // B = ... >>> + // ... = B >>> + // This still has an anti-dependence on B, but at least it isn't on the >>> + // original critical path. >>> + // >>> + // TODO: If we tracked more than one register here, we could potentially >>> + // fix that remaining critical edge too. This is a little more involved, >>> + // because unlike the most recent register, less recent registers should >>> + // still be considered, though only if no other registers are available. >>> + unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; >>> + >>> + // Attempt to break anti-dependence edges on the critical path. Walk the >>> + // instructions from the bottom up, tracking information about liveness >>> + // as we go to help determine which registers are available. >>> + unsigned Broken = 0; >>> + unsigned Count = InsertPosIndex - 1; >>> + for (MachineBasicBlock::iterator I = End, E = Begin; >>> + I != E; --Count) { >>> + MachineInstr *MI = --I; >>> + >>> + // Check if this instruction has a dependence on the critical path that >>> + // is an anti-dependence that we may be able to break. If it is, set >>> + // AntiDepReg to the non-zero register associated with the anti-dependence. >>> + // >>> + // We limit our attention to the critical path as a heuristic to avoid >>> + // breaking anti-dependence edges that aren't going to significantly >>> + // impact the overall schedule. There are a limited number of registers >>> + // and we want to save them for the important edges. >>> + // >>> + // TODO: Instructions with multiple defs could have multiple >>> + // anti-dependencies. The current code here only knows how to break one >>> + // edge per instruction. Note that we'd have to be able to break all of >>> + // the anti-dependencies in an instruction in order to be effective. >>> + unsigned AntiDepReg = 0; >>> + if (MI == CriticalPathMI) { >>> + if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >>> + SUnit *NextSU = Edge->getSUnit(); >>> + >>> + // Only consider anti-dependence edges. >>> + if (Edge->getKind() == SDep::Anti) { >>> + AntiDepReg = Edge->getReg(); >>> + assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >>> + if (!AllocatableSet.test(AntiDepReg)) >>> + // Don't break anti-dependencies on non-allocatable registers. >>> + AntiDepReg = 0; >>> + else if (KeepRegs.count(AntiDepReg)) >>> + // Don't break anti-dependencies if an use down below requires >>> + // this exact register. >>> + AntiDepReg = 0; >>> + else { >>> + // If the SUnit has other dependencies on the SUnit that it >>> + // anti-depends on, don't bother breaking the anti-dependency >>> + // since those edges would prevent such units from being >>> + // scheduled past each other regardless. >>> + // >>> + // Also, if there are dependencies on other SUnits with the >>> + // same register as the anti-dependency, don't attempt to >>> + // break it. >>> + for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), >>> + PE = CriticalPathSU->Preds.end(); P != PE; ++P) >>> + if (P->getSUnit() == NextSU ? >>> + (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : >>> + (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { >>> + AntiDepReg = 0; >>> + break; >>> + } >>> + } >>> + } >>> + CriticalPathSU = NextSU; >>> + CriticalPathMI = CriticalPathSU->getInstr(); >>> + } else { >>> + // We've reached the end of the critical path. >>> + CriticalPathSU = 0; >>> + CriticalPathMI = 0; >>> + } >>> + } >>> + >>> + PrescanInstruction(MI); >>> + >>> + if (MI->getDesc().hasExtraDefRegAllocReq()) >>> + // If this instruction's defs have special allocation requirement, don't >>> + // break this anti-dependency. >>> + AntiDepReg = 0; >>> + else if (AntiDepReg) { >>> + // If this instruction has a use of AntiDepReg, breaking it >>> + // is invalid. >>> + for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> + MachineOperand &MO = MI->getOperand(i); >>> + if (!MO.isReg()) continue; >>> + unsigned Reg = MO.getReg(); >>> + if (Reg == 0) continue; >>> + if (MO.isUse() && AntiDepReg == Reg) { >>> + AntiDepReg = 0; >>> + break; >>> + } >>> + } >>> + } >>> + >>> + // Determine AntiDepReg's register class, if it is live and is >>> + // consistently used within a single class. >>> + const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; >>> + assert((AntiDepReg == 0 || RC != NULL) && >>> + "Register should be live if it's causing an anti-dependence!"); >>> + if (RC == reinterpret_cast(-1)) >>> + AntiDepReg = 0; >>> + >>> + // Look for a suitable register to use to break the anti-depenence. >>> + // >>> + // TODO: Instead of picking the first free register, consider which might >>> + // be the best. >>> + if (AntiDepReg != 0) { >>> + if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >>> + LastNewReg[AntiDepReg], >>> + RC)) { >>> + DEBUG(errs() << "Breaking anti-dependence edge on " >>> + << TRI->getName(AntiDepReg) >>> + << " with " << RegRefs.count(AntiDepReg) << " references" >>> + << " using " << TRI->getName(NewReg) << "!\n"); >>> + >>> + // Update the references to the old register to refer to the new >>> + // register. >>> + std::pair::iterator, >>> + std::multimap::iterator> >>> + Range = RegRefs.equal_range(AntiDepReg); >>> + for (std::multimap::iterator >>> + Q = Range.first, QE = Range.second; Q != QE; ++Q) >>> + Q->second->setReg(NewReg); >>> + >>> + // We just went back in time and modified history; the >>> + // liveness information for the anti-depenence reg is now >>> + // inconsistent. Set the state as if it were dead. >>> + Classes[NewReg] = Classes[AntiDepReg]; >>> + DefIndices[NewReg] = DefIndices[AntiDepReg]; >>> + KillIndices[NewReg] = KillIndices[AntiDepReg]; >>> + assert(((KillIndices[NewReg] == ~0u) != >>> + (DefIndices[NewReg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for NewReg!"); >>> + >>> + Classes[AntiDepReg] = 0; >>> + DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >>> + KillIndices[AntiDepReg] = ~0u; >>> + assert(((KillIndices[AntiDepReg] == ~0u) != >>> + (DefIndices[AntiDepReg] == ~0u)) && >>> + "Kill and Def maps aren't consistent for AntiDepReg!"); >>> + >>> + RegRefs.erase(AntiDepReg); >>> + LastNewReg[AntiDepReg] = NewReg; >>> + ++Broken; >>> + } >>> + } >>> + >>> + ScanInstruction(MI, Count); >>> + } >>> + >>> + return Broken; >>> +} >>> >>> Added: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85127&view=auto >>> >>> ============================================================================== >>> --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (added) >>> +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 11:59:04 2009 >>> @@ -0,0 +1,95 @@ >>> +//=- llvm/CodeGen/CriticalAntiDepBreaker.h - Anti-Dep Support -*- C++ -*-=// >>> +// >>> +// The LLVM Compiler Infrastructure >>> +// >>> +// This file is distributed under the University of Illinois Open Source >>> +// License. See LICENSE.TXT for details. >>> +// >>> +//===----------------------------------------------------------------------===// >>> +// >>> +// This file implements the CriticalAntiDepBreaker class, which >>> +// implements register anti-dependence breaking along a blocks >>> +// critical path during post-RA scheduler. >>> +// >>> +//===----------------------------------------------------------------------===// >>> + >>> +#ifndef LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >>> +#define LLVM_CODEGEN_CRITICALANTIDEPBREAKER_H >>> + >>> +#include "llvm/CodeGen/AntiDepBreaker.h" >>> +#include "llvm/CodeGen/MachineBasicBlock.h" >>> +#include "llvm/CodeGen/MachineFrameInfo.h" >>> +#include "llvm/CodeGen/MachineFunction.h" >>> +#include "llvm/CodeGen/MachineRegisterInfo.h" >>> +#include "llvm/CodeGen/ScheduleDAG.h" >>> +#include "llvm/Target/TargetRegisterInfo.h" >>> +#include "llvm/ADT/BitVector.h" >>> +#include "llvm/ADT/SmallSet.h" >>> + >>> +namespace llvm { >>> + class CriticalAntiDepBreaker : public AntiDepBreaker { >>> + MachineFunction& MF; >>> + MachineRegisterInfo &MRI; >>> + const TargetRegisterInfo *TRI; >>> + >>> + /// AllocatableSet - The set of allocatable registers. >>> + /// We'll be ignoring anti-dependencies on non-allocatable registers, >>> + /// because they may not be safe to break. >>> + const BitVector AllocatableSet; >>> + >>> + /// Classes - For live regs that are only used in one register class in a >>> + /// live range, the register class. If the register is not live, the >>> + /// corresponding value is null. If the register is live but used in >>> + /// multiple register classes, the corresponding value is -1 casted to a >>> + /// pointer. >>> + const TargetRegisterClass * >>> + Classes[TargetRegisterInfo::FirstVirtualRegister]; >>> + >>> + /// RegRegs - Map registers to all their references within a live range. >>> + std::multimap RegRefs; >>> + >>> + /// KillIndices - The index of the most recent kill (proceding bottom-up), >>> + /// or ~0u if the register is not live. >>> + unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >>> + >>> + /// DefIndices - The index of the most recent complete def (proceding bottom >>> + /// up), or ~0u if the register is live. >>> + unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >>> + >>> + /// KeepRegs - A set of registers which are live and cannot be changed to >>> + /// break anti-dependencies. >>> + SmallSet KeepRegs; >>> + >>> + public: >>> + CriticalAntiDepBreaker(MachineFunction& MFi); >>> + ~CriticalAntiDepBreaker(); >>> + >>> + /// Start - Initialize anti-dep breaking for a new basic block. >>> + void StartBlock(MachineBasicBlock *BB); >>> + >>> + /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path >>> + /// of the ScheduleDAG and break them by renaming registers. >>> + /// >>> + unsigned BreakAntiDependencies(std::vector& SUnits, >>> + MachineBasicBlock::iterator& Begin, >>> + MachineBasicBlock::iterator& End, >>> + unsigned InsertPosIndex); >>> + >>> + /// Observe - Update liveness information to account for the current >>> + /// instruction, which will not be scheduled. >>> + /// >>> + void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); >>> + >>> + /// Finish - Finish anti-dep breaking for a basic block. >>> + void FinishBlock(); >>> + >>> + private: >>> + void PrescanInstruction(MachineInstr *MI); >>> + void ScanInstruction(MachineInstr *MI, unsigned Count); >>> + unsigned findSuitableFreeRegister(unsigned AntiDepReg, >>> + unsigned LastNewReg, >>> + const TargetRegisterClass *); >>> + }; >>> +} >>> + >>> +#endif >>> >>> Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85127&r1=85126&r2=85127&view=diff >>> >>> ============================================================================== >>> --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) >>> +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 11:59:04 2009 >>> @@ -19,6 +19,7 @@ >>> //===----------------------------------------------------------------------===// >>> >>> #define DEBUG_TYPE "post-RA-sched" >>> +#include "CriticalAntiDepBreaker.h" >>> #include "ExactHazardRecognizer.h" >>> #include "SimpleHazardRecognizer.h" >>> #include "ScheduleDAGInstrs.h" >>> @@ -40,6 +41,7 @@ >>> #include "llvm/Support/Debug.h" >>> #include "llvm/Support/ErrorHandling.h" >>> #include "llvm/Support/raw_ostream.h" >>> +#include "llvm/ADT/BitVector.h" >>> #include "llvm/ADT/Statistic.h" >>> #include >>> #include >>> @@ -47,6 +49,7 @@ >>> >>> STATISTIC(NumNoops, "Number of noops inserted"); >>> STATISTIC(NumStalls, "Number of pipeline stalls"); >>> +STATISTIC(NumFixedAnti, "Number of fixed anti-dependencies"); >>> >>> // Post-RA scheduling is enabled with >>> // TargetSubtarget.enablePostRAScheduler(). This flag can be used to >>> @@ -55,10 +58,11 @@ >>> EnablePostRAScheduler("post-RA-scheduler", >>> cl::desc("Enable scheduling after register allocation"), >>> cl::init(false), cl::Hidden); >>> -static cl::opt >>> +static cl::opt >>> EnableAntiDepBreaking("break-anti-dependencies", >>> - cl::desc("Break post-RA scheduling anti-dependencies"), >>> - cl::init(true), cl::Hidden); >>> + cl::desc("Break post-RA scheduling anti-dependencies: " >>> + "\"critical\", \"all\", or \"none\""), >>> + cl::init("none"), cl::Hidden); >>> static cl::opt >>> EnablePostRAHazardAvoidance("avoid-hazards", >>> cl::desc("Enable exact hazard avoidance"), >>> @@ -116,56 +120,30 @@ >>> /// Topo - A topological ordering for SUnits. >>> ScheduleDAGTopologicalSort Topo; >>> >>> - /// AllocatableSet - The set of allocatable registers. >>> - /// We'll be ignoring anti-dependencies on non-allocatable registers, >>> - /// because they may not be safe to break. >>> - const BitVector AllocatableSet; >>> - >>> /// HazardRec - The hazard recognizer to use. >>> ScheduleHazardRecognizer *HazardRec; >>> >>> + /// AntiDepBreak - Anti-dependence breaking object, or NULL if none >>> + AntiDepBreaker *AntiDepBreak; >>> + >>> /// AA - AliasAnalysis for making memory reference queries. >>> AliasAnalysis *AA; >>> >>> - /// AntiDepMode - Anti-dependence breaking mode >>> - TargetSubtarget::AntiDepBreakMode AntiDepMode; >>> - >>> - /// Classes - For live regs that are only used in one register class in a >>> - /// live range, the register class. If the register is not live, the >>> - /// corresponding value is null. If the register is live but used in >>> - /// multiple register classes, the corresponding value is -1 casted to a >>> - /// pointer. >>> - const TargetRegisterClass * >>> - Classes[TargetRegisterInfo::FirstVirtualRegister]; >>> - >>> - /// RegRegs - Map registers to all their references within a live range. >>> - std::multimap RegRefs; >>> - >>> /// KillIndices - The index of the most recent kill (proceding bottom-up), >>> /// or ~0u if the register is not live. >>> unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; >>> >>> - /// DefIndices - The index of the most recent complete def (proceding bottom >>> - /// up), or ~0u if the register is live. >>> - unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; >>> - >>> - /// KeepRegs - A set of registers which are live and cannot be changed to >>> - /// break anti-dependencies. >>> - SmallSet KeepRegs; >>> - >>> public: >>> SchedulePostRATDList(MachineFunction &MF, >>> const MachineLoopInfo &MLI, >>> const MachineDominatorTree &MDT, >>> ScheduleHazardRecognizer *HR, >>> - AliasAnalysis *aa, >>> - TargetSubtarget::AntiDepBreakMode adm) >>> + AntiDepBreaker *ADB, >>> + AliasAnalysis *aa) >>> : ScheduleDAGInstrs(MF, MLI, MDT), Topo(SUnits), >>> - AllocatableSet(TRI->getAllocatableSet(MF)), >>> - HazardRec(HR), AA(aa), AntiDepMode(adm) {} >>> + HazardRec(HR), AntiDepBreak(ADB), AA(aa) {} >>> >>> ~SchedulePostRATDList() { >>> - delete HazardRec; >>> } >>> >>> /// StartBlock - Initialize register live-range state for scheduling in >>> @@ -177,11 +155,6 @@ >>> /// >>> void Schedule(); >>> >>> - /// FixupKills - Fix register kill flags that have been made >>> - /// invalid due to scheduling >>> - /// >>> - void FixupKills(MachineBasicBlock *MBB); >>> - >>> /// Observe - Update liveness information to account for the current >>> /// instruction, which will not be scheduled. >>> /// >>> @@ -191,17 +164,16 @@ >>> /// >>> void FinishBlock(); >>> >>> + /// FixupKills - Fix register kill flags that have been made >>> + /// invalid due to scheduling >>> + /// >>> + void FixupKills(MachineBasicBlock *MBB); >>> + >>> private: >>> - void PrescanInstruction(MachineInstr *MI); >>> - void ScanInstruction(MachineInstr *MI, unsigned Count); >>> void ReleaseSucc(SUnit *SU, SDep *SuccEdge); >>> void ReleaseSuccessors(SUnit *SU); >>> void ScheduleNodeTopDown(SUnit *SU, unsigned CurCycle); >>> void ListScheduleTopDown(); >>> - bool BreakAntiDependencies(); >>> - unsigned findSuitableFreeRegister(unsigned AntiDepReg, >>> - unsigned LastNewReg, >>> - const TargetRegisterClass *); >>> void StartBlockForKills(MachineBasicBlock *BB); >>> >>> // ToggleKillFlag - Toggle a register operand kill flag. Other >>> @@ -250,8 +222,9 @@ >>> >>> // Check for antidep breaking override... >>> if (EnableAntiDepBreaking.getPosition() > 0) { >>> - AntiDepMode = (EnableAntiDepBreaking) ? >>> - TargetSubtarget::ANTIDEP_CRITICAL : TargetSubtarget::ANTIDEP_NONE; >>> + AntiDepMode = (EnableAntiDepBreaking == "all") ? TargetSubtarget::ANTIDEP_ALL : >>> + (EnableAntiDepBreaking == "critical") ? TargetSubtarget::ANTIDEP_CRITICAL : >>> + TargetSubtarget::ANTIDEP_NONE; >>> } >>> >>> DEBUG(errs() << "PostRAScheduler\n"); >>> @@ -262,8 +235,12 @@ >>> ScheduleHazardRecognizer *HR = EnablePostRAHazardAvoidance ? >>> (ScheduleHazardRecognizer *)new ExactHazardRecognizer(InstrItins) : >>> (ScheduleHazardRecognizer *)new SimpleHazardRecognizer(); >>> + AntiDepBreaker *ADB = >>> + (AntiDepMode == TargetSubtarget::ANTIDEP_ALL) ? NULL /* FIXME */ : >>> + (AntiDepMode == TargetSubtarget::ANTIDEP_CRITICAL) ? >>> + new CriticalAntiDepBreaker(Fn) : NULL; >>> >>> - SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, AA, AntiDepMode); >>> + SchedulePostRATDList Scheduler(Fn, MLI, MDT, HR, ADB, AA); >>> >>> // Loop over all of the basic blocks >>> for (MachineFunction::iterator MBB = Fn.begin(), MBBe = Fn.end(); >>> @@ -311,6 +288,9 @@ >>> Scheduler.FixupKills(MBB); >>> } >>> >>> + delete HR; >>> + delete ADB; >>> + >>> return true; >>> } >>> >>> @@ -321,78 +301,10 @@ >>> // Call the superclass. >>> ScheduleDAGInstrs::StartBlock(BB); >>> >>> - // Reset the hazard recognizer. >>> + // Reset the hazard recognizer and anti-dep breaker. >>> HazardRec->Reset(); >>> - >>> - // Clear out the register class data. >>> - std::fill(Classes, array_endof(Classes), >>> - static_cast(0)); >>> - >>> - // Initialize the indices to indicate that no registers are live. >>> - std::fill(KillIndices, array_endof(KillIndices), ~0u); >>> - std::fill(DefIndices, array_endof(DefIndices), BB->size()); >>> - >>> - // Clear "do not change" set. >>> - KeepRegs.clear(); >>> - >>> - bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); >>> - >>> - // Determine the live-out physregs for this block. >>> - if (IsReturnBlock) { >>> - // In a return block, examine the function live-out regs. >>> - for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), >>> - E = MRI.liveout_end(); I != E; ++I) { >>> - unsigned Reg = *I; >>> - Classes[Reg] = reinterpret_cast(-1); >>> - KillIndices[Reg] = BB->size(); >>> - DefIndices[Reg] = ~0u; >>> - // Repeat, for all aliases. >>> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> - unsigned AliasReg = *Alias; >>> - Classes[AliasReg] = reinterpret_cast(-1); >>> - KillIndices[AliasReg] = BB->size(); >>> - DefIndices[AliasReg] = ~0u; >>> - } >>> - } >>> - } else { >>> - // In a non-return block, examine the live-in regs of all successors. >>> - for (MachineBasicBlock::succ_iterator SI = BB->succ_begin(), >>> - SE = BB->succ_end(); SI != SE; ++SI) >>> - for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), >>> - E = (*SI)->livein_end(); I != E; ++I) { >>> - unsigned Reg = *I; >>> - Classes[Reg] = reinterpret_cast(-1); >>> - KillIndices[Reg] = BB->size(); >>> - DefIndices[Reg] = ~0u; >>> - // Repeat, for all aliases. >>> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> - unsigned AliasReg = *Alias; >>> - Classes[AliasReg] = reinterpret_cast(-1); >>> - KillIndices[AliasReg] = BB->size(); >>> - DefIndices[AliasReg] = ~0u; >>> - } >>> - } >>> - } >>> - >>> - // Mark live-out callee-saved registers. In a return block this is >>> - // all callee-saved registers. In non-return this is any >>> - // callee-saved register that is not saved in the prolog. >>> - const MachineFrameInfo *MFI = MF.getFrameInfo(); >>> - BitVector Pristine = MFI->getPristineRegs(BB); >>> - for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { >>> - unsigned Reg = *I; >>> - if (!IsReturnBlock && !Pristine.test(Reg)) continue; >>> - Classes[Reg] = reinterpret_cast(-1); >>> - KillIndices[Reg] = BB->size(); >>> - DefIndices[Reg] = ~0u; >>> - // Repeat, for all aliases. >>> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> - unsigned AliasReg = *Alias; >>> - Classes[AliasReg] = reinterpret_cast(-1); >>> - KillIndices[AliasReg] = BB->size(); >>> - DefIndices[AliasReg] = ~0u; >>> - } >>> - } >>> + if (AntiDepBreak != NULL) >>> + AntiDepBreak->StartBlock(BB); >>> } >>> >>> /// Schedule - Schedule the instruction range using list scheduling. >>> @@ -403,8 +315,11 @@ >>> // Build the scheduling graph. >>> BuildSchedGraph(AA); >>> >>> - if (AntiDepMode != TargetSubtarget::ANTIDEP_NONE) { >>> - if (BreakAntiDependencies()) { >>> + if (AntiDepBreak != NULL) { >>> + unsigned Broken = >>> + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, >>> + InsertPosIndex); >>> + if (Broken > 0) { >>> // We made changes. Update the dependency graph. >>> // Theoretically we could update the graph in place: >>> // When a live range is changed to use a different register, remove >>> @@ -415,6 +330,8 @@ >>> EntrySU = SUnit(); >>> ExitSU = SUnit(); >>> BuildSchedGraph(AA); >>> + >>> + NumFixedAnti += Broken; >>> } >>> } >>> >>> @@ -432,436 +349,20 @@ >>> /// instruction, which will not be scheduled. >>> /// >>> void SchedulePostRATDList::Observe(MachineInstr *MI, unsigned Count) { >>> - assert(Count < InsertPosIndex && "Instruction index out of expected range!"); >>> - >>> - // Any register which was defined within the previous scheduling region >>> - // may have been rescheduled and its lifetime may overlap with registers >>> - // in ways not reflected in our current liveness state. For each such >>> - // register, adjust the liveness state to be conservatively correct. >>> - for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) >>> - if (DefIndices[Reg] < InsertPosIndex && DefIndices[Reg] >= Count) { >>> - assert(KillIndices[Reg] == ~0u && "Clobbered register is live!"); >>> - // Mark this register to be non-renamable. >>> - Classes[Reg] = reinterpret_cast(-1); >>> - // Move the def index to the end of the previous region, to reflect >>> - // that the def could theoretically have been scheduled at the end. >>> - DefIndices[Reg] = InsertPosIndex; >>> - } >>> - >>> - PrescanInstruction(MI); >>> - ScanInstruction(MI, Count); >>> + if (AntiDepBreak != NULL) >>> + AntiDepBreak->Observe(MI, Count, InsertPosIndex); >>> } >>> >>> /// FinishBlock - Clean up register live-range state. >>> /// >>> void SchedulePostRATDList::FinishBlock() { >>> - RegRefs.clear(); >>> + if (AntiDepBreak != NULL) >>> + AntiDepBreak->FinishBlock(); >>> >>> // Call the superclass. >>> ScheduleDAGInstrs::FinishBlock(); >>> } >>> >>> -/// CriticalPathStep - Return the next SUnit after SU on the bottom-up >>> -/// critical path. >>> -static SDep *CriticalPathStep(SUnit *SU) { >>> - SDep *Next = 0; >>> - unsigned NextDepth = 0; >>> - // Find the predecessor edge with the greatest depth. >>> - for (SUnit::pred_iterator P = SU->Preds.begin(), PE = SU->Preds.end(); >>> - P != PE; ++P) { >>> - SUnit *PredSU = P->getSUnit(); >>> - unsigned PredLatency = P->getLatency(); >>> - unsigned PredTotalLatency = PredSU->getDepth() + PredLatency; >>> - // In the case of a latency tie, prefer an anti-dependency edge over >>> - // other types of edges. >>> - if (NextDepth < PredTotalLatency || >>> - (NextDepth == PredTotalLatency && P->getKind() == SDep::Anti)) { >>> - NextDepth = PredTotalLatency; >>> - Next = &*P; >>> - } >>> - } >>> - return Next; >>> -} >>> - >>> -void SchedulePostRATDList::PrescanInstruction(MachineInstr *MI) { >>> - // Scan the register operands for this instruction and update >>> - // Classes and RegRefs. >>> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> - MachineOperand &MO = MI->getOperand(i); >>> - if (!MO.isReg()) continue; >>> - unsigned Reg = MO.getReg(); >>> - if (Reg == 0) continue; >>> - const TargetRegisterClass *NewRC = 0; >>> - >>> - if (i < MI->getDesc().getNumOperands()) >>> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >>> - >>> - // For now, only allow the register to be changed if its register >>> - // class is consistent across all uses. >>> - if (!Classes[Reg] && NewRC) >>> - Classes[Reg] = NewRC; >>> - else if (!NewRC || Classes[Reg] != NewRC) >>> - Classes[Reg] = reinterpret_cast(-1); >>> - >>> - // Now check for aliases. >>> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> - // If an alias of the reg is used during the live range, give up. >>> - // Note that this allows us to skip checking if AntiDepReg >>> - // overlaps with any of the aliases, among other things. >>> - unsigned AliasReg = *Alias; >>> - if (Classes[AliasReg]) { >>> - Classes[AliasReg] = reinterpret_cast(-1); >>> - Classes[Reg] = reinterpret_cast(-1); >>> - } >>> - } >>> - >>> - // If we're still willing to consider this register, note the reference. >>> - if (Classes[Reg] != reinterpret_cast(-1)) >>> - RegRefs.insert(std::make_pair(Reg, &MO)); >>> - >>> - // It's not safe to change register allocation for source operands of >>> - // that have special allocation requirements. >>> - if (MO.isUse() && MI->getDesc().hasExtraSrcRegAllocReq()) { >>> - if (KeepRegs.insert(Reg)) { >>> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >>> - *Subreg; ++Subreg) >>> - KeepRegs.insert(*Subreg); >>> - } >>> - } >>> - } >>> -} >>> - >>> -void SchedulePostRATDList::ScanInstruction(MachineInstr *MI, >>> - unsigned Count) { >>> - // Update liveness. >>> - // Proceding upwards, registers that are defed but not used in this >>> - // instruction are now dead. >>> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> - MachineOperand &MO = MI->getOperand(i); >>> - if (!MO.isReg()) continue; >>> - unsigned Reg = MO.getReg(); >>> - if (Reg == 0) continue; >>> - if (!MO.isDef()) continue; >>> - // Ignore two-addr defs. >>> - if (MI->isRegTiedToUseOperand(i)) continue; >>> - >>> - DefIndices[Reg] = Count; >>> - KillIndices[Reg] = ~0u; >>> - assert(((KillIndices[Reg] == ~0u) != >>> - (DefIndices[Reg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for Reg!"); >>> - KeepRegs.erase(Reg); >>> - Classes[Reg] = 0; >>> - RegRefs.erase(Reg); >>> - // Repeat, for all subregs. >>> - for (const unsigned *Subreg = TRI->getSubRegisters(Reg); >>> - *Subreg; ++Subreg) { >>> - unsigned SubregReg = *Subreg; >>> - DefIndices[SubregReg] = Count; >>> - KillIndices[SubregReg] = ~0u; >>> - KeepRegs.erase(SubregReg); >>> - Classes[SubregReg] = 0; >>> - RegRefs.erase(SubregReg); >>> - } >>> - // Conservatively mark super-registers as unusable. >>> - for (const unsigned *Super = TRI->getSuperRegisters(Reg); >>> - *Super; ++Super) { >>> - unsigned SuperReg = *Super; >>> - Classes[SuperReg] = reinterpret_cast(-1); >>> - } >>> - } >>> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> - MachineOperand &MO = MI->getOperand(i); >>> - if (!MO.isReg()) continue; >>> - unsigned Reg = MO.getReg(); >>> - if (Reg == 0) continue; >>> - if (!MO.isUse()) continue; >>> - >>> - const TargetRegisterClass *NewRC = 0; >>> - if (i < MI->getDesc().getNumOperands()) >>> - NewRC = MI->getDesc().OpInfo[i].getRegClass(TRI); >>> - >>> - // For now, only allow the register to be changed if its register >>> - // class is consistent across all uses. >>> - if (!Classes[Reg] && NewRC) >>> - Classes[Reg] = NewRC; >>> - else if (!NewRC || Classes[Reg] != NewRC) >>> - Classes[Reg] = reinterpret_cast(-1); >>> - >>> - RegRefs.insert(std::make_pair(Reg, &MO)); >>> - >>> - // It wasn't previously live but now it is, this is a kill. >>> - if (KillIndices[Reg] == ~0u) { >>> - KillIndices[Reg] = Count; >>> - DefIndices[Reg] = ~0u; >>> - assert(((KillIndices[Reg] == ~0u) != >>> - (DefIndices[Reg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for Reg!"); >>> - } >>> - // Repeat, for all aliases. >>> - for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { >>> - unsigned AliasReg = *Alias; >>> - if (KillIndices[AliasReg] == ~0u) { >>> - KillIndices[AliasReg] = Count; >>> - DefIndices[AliasReg] = ~0u; >>> - } >>> - } >>> - } >>> -} >>> - >>> -unsigned >>> -SchedulePostRATDList::findSuitableFreeRegister(unsigned AntiDepReg, >>> - unsigned LastNewReg, >>> - const TargetRegisterClass *RC) { >>> - for (TargetRegisterClass::iterator R = RC->allocation_order_begin(MF), >>> - RE = RC->allocation_order_end(MF); R != RE; ++R) { >>> - unsigned NewReg = *R; >>> - // Don't replace a register with itself. >>> - if (NewReg == AntiDepReg) continue; >>> - // Don't replace a register with one that was recently used to repair >>> - // an anti-dependence with this AntiDepReg, because that would >>> - // re-introduce that anti-dependence. >>> - if (NewReg == LastNewReg) continue; >>> - // If NewReg is dead and NewReg's most recent def is not before >>> - // AntiDepReg's kill, it's safe to replace AntiDepReg with NewReg. >>> - assert(((KillIndices[AntiDepReg] == ~0u) != (DefIndices[AntiDepReg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for AntiDepReg!"); >>> - assert(((KillIndices[NewReg] == ~0u) != (DefIndices[NewReg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for NewReg!"); >>> - if (KillIndices[NewReg] != ~0u || >>> - Classes[NewReg] == reinterpret_cast(-1) || >>> - KillIndices[AntiDepReg] > DefIndices[NewReg]) >>> - continue; >>> - return NewReg; >>> - } >>> - >>> - // No registers are free and available! >>> - return 0; >>> -} >>> - >>> -/// BreakAntiDependencies - Identifiy anti-dependencies along the critical path >>> -/// of the ScheduleDAG and break them by renaming registers. >>> -/// >>> -bool SchedulePostRATDList::BreakAntiDependencies() { >>> - // The code below assumes that there is at least one instruction, >>> - // so just duck out immediately if the block is empty. >>> - if (SUnits.empty()) return false; >>> - >>> - // Find the node at the bottom of the critical path. >>> - SUnit *Max = 0; >>> - for (unsigned i = 0, e = SUnits.size(); i != e; ++i) { >>> - SUnit *SU = &SUnits[i]; >>> - if (!Max || SU->getDepth() + SU->Latency > Max->getDepth() + Max->Latency) >>> - Max = SU; >>> - } >>> - >>> -#ifndef NDEBUG >>> - { >>> - DEBUG(errs() << "Critical path has total latency " >>> - << (Max->getDepth() + Max->Latency) << "\n"); >>> - DEBUG(errs() << "Available regs:"); >>> - for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { >>> - if (KillIndices[Reg] == ~0u) >>> - DEBUG(errs() << " " << TRI->getName(Reg)); >>> - } >>> - DEBUG(errs() << '\n'); >>> - } >>> -#endif >>> - >>> - // Track progress along the critical path through the SUnit graph as we walk >>> - // the instructions. >>> - SUnit *CriticalPathSU = Max; >>> - MachineInstr *CriticalPathMI = CriticalPathSU->getInstr(); >>> - >>> - // Consider this pattern: >>> - // A = ... >>> - // ... = A >>> - // A = ... >>> - // ... = A >>> - // A = ... >>> - // ... = A >>> - // A = ... >>> - // ... = A >>> - // There are three anti-dependencies here, and without special care, >>> - // we'd break all of them using the same register: >>> - // A = ... >>> - // ... = A >>> - // B = ... >>> - // ... = B >>> - // B = ... >>> - // ... = B >>> - // B = ... >>> - // ... = B >>> - // because at each anti-dependence, B is the first register that >>> - // isn't A which is free. This re-introduces anti-dependencies >>> - // at all but one of the original anti-dependencies that we were >>> - // trying to break. To avoid this, keep track of the most recent >>> - // register that each register was replaced with, avoid >>> - // using it to repair an anti-dependence on the same register. >>> - // This lets us produce this: >>> - // A = ... >>> - // ... = A >>> - // B = ... >>> - // ... = B >>> - // C = ... >>> - // ... = C >>> - // B = ... >>> - // ... = B >>> - // This still has an anti-dependence on B, but at least it isn't on the >>> - // original critical path. >>> - // >>> - // TODO: If we tracked more than one register here, we could potentially >>> - // fix that remaining critical edge too. This is a little more involved, >>> - // because unlike the most recent register, less recent registers should >>> - // still be considered, though only if no other registers are available. >>> - unsigned LastNewReg[TargetRegisterInfo::FirstVirtualRegister] = {}; >>> - >>> - // Attempt to break anti-dependence edges on the critical path. Walk the >>> - // instructions from the bottom up, tracking information about liveness >>> - // as we go to help determine which registers are available. >>> - bool Changed = false; >>> - unsigned Count = InsertPosIndex - 1; >>> - for (MachineBasicBlock::iterator I = InsertPos, E = Begin; >>> - I != E; --Count) { >>> - MachineInstr *MI = --I; >>> - >>> - // Check if this instruction has a dependence on the critical path that >>> - // is an anti-dependence that we may be able to break. If it is, set >>> - // AntiDepReg to the non-zero register associated with the anti-dependence. >>> - // >>> - // We limit our attention to the critical path as a heuristic to avoid >>> - // breaking anti-dependence edges that aren't going to significantly >>> - // impact the overall schedule. There are a limited number of registers >>> - // and we want to save them for the important edges. >>> - // >>> - // TODO: Instructions with multiple defs could have multiple >>> - // anti-dependencies. The current code here only knows how to break one >>> - // edge per instruction. Note that we'd have to be able to break all of >>> - // the anti-dependencies in an instruction in order to be effective. >>> - unsigned AntiDepReg = 0; >>> - if (MI == CriticalPathMI) { >>> - if (SDep *Edge = CriticalPathStep(CriticalPathSU)) { >>> - SUnit *NextSU = Edge->getSUnit(); >>> - >>> - // Only consider anti-dependence edges. >>> - if (Edge->getKind() == SDep::Anti) { >>> - AntiDepReg = Edge->getReg(); >>> - assert(AntiDepReg != 0 && "Anti-dependence on reg0?"); >>> - if (!AllocatableSet.test(AntiDepReg)) >>> - // Don't break anti-dependencies on non-allocatable registers. >>> - AntiDepReg = 0; >>> - else if (KeepRegs.count(AntiDepReg)) >>> - // Don't break anti-dependencies if an use down below requires >>> - // this exact register. >>> - AntiDepReg = 0; >>> - else { >>> - // If the SUnit has other dependencies on the SUnit that it >>> - // anti-depends on, don't bother breaking the anti-dependency >>> - // since those edges would prevent such units from being >>> - // scheduled past each other regardless. >>> - // >>> - // Also, if there are dependencies on other SUnits with the >>> - // same register as the anti-dependency, don't attempt to >>> - // break it. >>> - for (SUnit::pred_iterator P = CriticalPathSU->Preds.begin(), >>> - PE = CriticalPathSU->Preds.end(); P != PE; ++P) >>> - if (P->getSUnit() == NextSU ? >>> - (P->getKind() != SDep::Anti || P->getReg() != AntiDepReg) : >>> - (P->getKind() == SDep::Data && P->getReg() == AntiDepReg)) { >>> - AntiDepReg = 0; >>> - break; >>> - } >>> - } >>> - } >>> - CriticalPathSU = NextSU; >>> - CriticalPathMI = CriticalPathSU->getInstr(); >>> - } else { >>> - // We've reached the end of the critical path. >>> - CriticalPathSU = 0; >>> - CriticalPathMI = 0; >>> - } >>> - } >>> - >>> - PrescanInstruction(MI); >>> - >>> - if (MI->getDesc().hasExtraDefRegAllocReq()) >>> - // If this instruction's defs have special allocation requirement, don't >>> - // break this anti-dependency. >>> - AntiDepReg = 0; >>> - else if (AntiDepReg) { >>> - // If this instruction has a use of AntiDepReg, breaking it >>> - // is invalid. >>> - for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { >>> - MachineOperand &MO = MI->getOperand(i); >>> - if (!MO.isReg()) continue; >>> - unsigned Reg = MO.getReg(); >>> - if (Reg == 0) continue; >>> - if (MO.isUse() && AntiDepReg == Reg) { >>> - AntiDepReg = 0; >>> - break; >>> - } >>> - } >>> - } >>> - >>> - // Determine AntiDepReg's register class, if it is live and is >>> - // consistently used within a single class. >>> - const TargetRegisterClass *RC = AntiDepReg != 0 ? Classes[AntiDepReg] : 0; >>> - assert((AntiDepReg == 0 || RC != NULL) && >>> - "Register should be live if it's causing an anti-dependence!"); >>> - if (RC == reinterpret_cast(-1)) >>> - AntiDepReg = 0; >>> - >>> - // Look for a suitable register to use to break the anti-depenence. >>> - // >>> - // TODO: Instead of picking the first free register, consider which might >>> - // be the best. >>> - if (AntiDepReg != 0) { >>> - if (unsigned NewReg = findSuitableFreeRegister(AntiDepReg, >>> - LastNewReg[AntiDepReg], >>> - RC)) { >>> - DEBUG(errs() << "Breaking anti-dependence edge on " >>> - << TRI->getName(AntiDepReg) >>> - << " with " << RegRefs.count(AntiDepReg) << " references" >>> - << " using " << TRI->getName(NewReg) << "!\n"); >>> - >>> - // Update the references to the old register to refer to the new >>> - // register. >>> - std::pair::iterator, >>> - std::multimap::iterator> >>> - Range = RegRefs.equal_range(AntiDepReg); >>> - for (std::multimap::iterator >>> - Q = Range.first, QE = Range.second; Q != QE; ++Q) >>> - Q->second->setReg(NewReg); >>> - >>> - // We just went back in time and modified history; the >>> - // liveness information for the anti-depenence reg is now >>> - // inconsistent. Set the state as if it were dead. >>> - Classes[NewReg] = Classes[AntiDepReg]; >>> - DefIndices[NewReg] = DefIndices[AntiDepReg]; >>> - KillIndices[NewReg] = KillIndices[AntiDepReg]; >>> - assert(((KillIndices[NewReg] == ~0u) != >>> - (DefIndices[NewReg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for NewReg!"); >>> - >>> - Classes[AntiDepReg] = 0; >>> - DefIndices[AntiDepReg] = KillIndices[AntiDepReg]; >>> - KillIndices[AntiDepReg] = ~0u; >>> - assert(((KillIndices[AntiDepReg] == ~0u) != >>> - (DefIndices[AntiDepReg] == ~0u)) && >>> - "Kill and Def maps aren't consistent for AntiDepReg!"); >>> - >>> - RegRefs.erase(AntiDepReg); >>> - Changed = true; >>> - LastNewReg[AntiDepReg] = NewReg; >>> - } >>> - } >>> - >>> - ScanInstruction(MI, Count); >>> - } >>> - >>> - return Changed; >>> -} >>> - >>> /// StartBlockForKills - Initialize register live-range state for updating kills >>> /// >>> void SchedulePostRATDList::StartBlockForKills(MachineBasicBlock *BB) { >>> >>> Modified: llvm/trunk/lib/Target/ARM/ARMSubtarget.h >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMSubtarget.h?rev=85127&r1=85126&r2=85127&view=diff >>> >>> ============================================================================== >>> --- llvm/trunk/lib/Target/ARM/ARMSubtarget.h (original) >>> +++ llvm/trunk/lib/Target/ARM/ARMSubtarget.h Mon Oct 26 11:59:04 2009 >>> @@ -130,7 +130,7 @@ >>> /// for Thumb1. >>> bool enablePostRAScheduler(CodeGenOpt::Level OptLevel, >>> TargetSubtarget::AntiDepBreakMode& mode) const { >>> - mode = TargetSubtarget::ANTIDEP_NONE; >>> + mode = TargetSubtarget::ANTIDEP_CRITICAL; >>> return PostRAScheduler && OptLevel >= CodeGenOpt::Default; >>> } >>> >>> >>> Modified: llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll >>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll?rev=85127&r1=85126&r2=85127&view=diff >>> >>> ============================================================================== >>> --- llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll (original) >>> +++ llvm/trunk/test/CodeGen/X86/break-anti-dependencies.ll Mon Oct 26 11:59:04 2009 >>> @@ -1,7 +1,7 @@ >>> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=false > %t >>> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=none > %t >>> ; RUN: grep {%xmm0} %t | count 14 >>> ; RUN: not grep {%xmm1} %t >>> -; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies > %t >>> +; RUN: llc < %s -march=x86-64 -post-RA-scheduler -break-anti-dependencies=critical > %t >>> ; RUN: grep {%xmm0} %t | count 7 >>> ; RUN: grep {%xmm1} %t | count 7 >>> >>> >>> >>> _______________________________________________ >>> llvm-commits mailing list >>> llvm-commits at cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits >> > From johnny.chen at apple.com Mon Oct 26 16:39:13 2009 From: johnny.chen at apple.com (Johnny Chen) Date: Mon, 26 Oct 2009 14:39:13 -0700 Subject: [llvm-commits] Typo in ARMInstrFormats.td for ASuIn? Message-ID: <5E360501-8B49-454C-A0CC-0EE2FE75BB82@apple.com> Hi, Opcod3 is not passed to ASuI parent class. -------------- next part -------------- A non-text attachment was scrubbed... Name: ARMInstrFormats.td.patch Type: application/octet-stream Size: 784 bytes Desc: not available Url : http://lists.cs.uiuc.edu/pipermail/llvm-commits/attachments/20091026/4a66c9b8/attachment.obj From gohman at apple.com Mon Oct 26 16:55:44 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 21:55:44 -0000 Subject: [llvm-commits] [llvm] r85158 - in /llvm/trunk: lib/Analysis/BasicAliasAnalysis.cpp test/Analysis/BasicAA/phi-and-select.ll Message-ID: <200910262155.n9QLtiAA006376@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 16:55:43 2009 New Revision: 85158 URL: http://llvm.org/viewvc/llvm-project?rev=85158&view=rev Log: Teach BasicAA how to analyze Select instructions, and make it more aggressive on PHI instructions. Added: llvm/trunk/test/Analysis/BasicAA/phi-and-select.ll Modified: llvm/trunk/lib/Analysis/BasicAliasAnalysis.cpp Modified: llvm/trunk/lib/Analysis/BasicAliasAnalysis.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/BasicAliasAnalysis.cpp?rev=85158&r1=85157&r2=85158&view=diff ============================================================================== --- llvm/trunk/lib/Analysis/BasicAliasAnalysis.cpp (original) +++ llvm/trunk/lib/Analysis/BasicAliasAnalysis.cpp Mon Oct 26 16:55:43 2009 @@ -217,7 +217,7 @@ private: // VisitedPHIs - Track PHI nodes visited by a aliasCheck() call. - SmallPtrSet VisitedPHIs; + SmallPtrSet VisitedPHIs; // aliasGEP - Provide a bunch of ad-hoc rules to disambiguate a GEP instruction // against another. @@ -229,6 +229,10 @@ AliasResult aliasPHI(const PHINode *PN, unsigned PNSize, const Value *V2, unsigned V2Size); + /// aliasSelect - Disambiguate a Select instruction against another value. + AliasResult aliasSelect(const SelectInst *SI, unsigned SISize, + const Value *V2, unsigned V2Size); + AliasResult aliasCheck(const Value *V1, unsigned V1Size, const Value *V2, unsigned V2Size); @@ -519,6 +523,41 @@ return MayAlias; } +// aliasSelect - Provide a bunch of ad-hoc rules to disambiguate a Select instruction +// against another. +AliasAnalysis::AliasResult +BasicAliasAnalysis::aliasSelect(const SelectInst *SI, unsigned SISize, + const Value *V2, unsigned V2Size) { + // If the values are Selects with the same condition, we can do a more precise + // check: just check for aliases between the values on corresponding arms. + if (const SelectInst *SI2 = dyn_cast(V2)) + if (SI->getCondition() == SI2->getCondition()) { + AliasResult Alias = + aliasCheck(SI->getTrueValue(), SISize, + SI2->getTrueValue(), V2Size); + if (Alias == MayAlias) + return MayAlias; + AliasResult ThisAlias = + aliasCheck(SI->getFalseValue(), SISize, + SI2->getFalseValue(), V2Size); + if (ThisAlias != Alias) + return MayAlias; + return Alias; + } + + // If both arms of the Select node NoAlias or MustAlias V2, then returns + // NoAlias / MustAlias. Otherwise, returns MayAlias. + AliasResult Alias = + aliasCheck(SI->getTrueValue(), SISize, V2, V2Size); + if (Alias == MayAlias) + return MayAlias; + AliasResult ThisAlias = + aliasCheck(SI->getFalseValue(), SISize, V2, V2Size); + if (ThisAlias != Alias) + return MayAlias; + return Alias; +} + // aliasPHI - Provide a bunch of ad-hoc rules to disambiguate a PHI instruction // against another. AliasAnalysis::AliasResult @@ -528,6 +567,28 @@ if (!VisitedPHIs.insert(PN)) return MayAlias; + // If the values are PHIs in the same block, we can do a more precise + // as well as efficient check: just check for aliases between the values + // on corresponding edges. + if (const PHINode *PN2 = dyn_cast(V2)) + if (PN2->getParent() == PN->getParent()) { + AliasResult Alias = + aliasCheck(PN->getIncomingValue(0), PNSize, + PN2->getIncomingValueForBlock(PN->getIncomingBlock(0)), + V2Size); + if (Alias == MayAlias) + return MayAlias; + for (unsigned i = 1, e = PN->getNumIncomingValues(); i != e; ++i) { + AliasResult ThisAlias = + aliasCheck(PN->getIncomingValue(i), PNSize, + PN2->getIncomingValueForBlock(PN->getIncomingBlock(i)), + V2Size); + if (ThisAlias != Alias) + return MayAlias; + } + return Alias; + } + SmallPtrSet UniqueSrc; SmallVector V1Srcs; for (unsigned i = 0, e = PN->getNumIncomingValues(); i != e; ++i) { @@ -542,7 +603,7 @@ V1Srcs.push_back(PV1); } - AliasResult Alias = aliasCheck(V1Srcs[0], PNSize, V2, V2Size); + AliasResult Alias = aliasCheck(V2, V2Size, V1Srcs[0], PNSize); // Early exit if the check of the first PHI source against V2 is MayAlias. // Other results are not possible. if (Alias == MayAlias) @@ -552,6 +613,12 @@ // NoAlias / MustAlias. Otherwise, returns MayAlias. for (unsigned i = 1, e = V1Srcs.size(); i != e; ++i) { Value *V = V1Srcs[i]; + + // If V2 is a PHI, the recursive case will have been caught in the + // above aliasCheck call, so these subsequent calls to aliasCheck + // don't need to assume that V2 is being visited recursively. + VisitedPHIs.erase(V2); + AliasResult ThisAlias = aliasCheck(V2, V2Size, V, PNSize); if (ThisAlias != Alias || ThisAlias == MayAlias) return MayAlias; @@ -628,6 +695,13 @@ if (const PHINode *PN = dyn_cast(V1)) return aliasPHI(PN, V1Size, V2, V2Size); + if (isa(V2) && !isa(V1)) { + std::swap(V1, V2); + std::swap(V1Size, V2Size); + } + if (const SelectInst *S1 = dyn_cast(V1)) + return aliasSelect(S1, V1Size, V2, V2Size); + return MayAlias; } Added: llvm/trunk/test/Analysis/BasicAA/phi-and-select.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Analysis/BasicAA/phi-and-select.ll?rev=85158&view=auto ============================================================================== --- llvm/trunk/test/Analysis/BasicAA/phi-and-select.ll (added) +++ llvm/trunk/test/Analysis/BasicAA/phi-and-select.ll Mon Oct 26 16:55:43 2009 @@ -0,0 +1,73 @@ +; RUN: opt < %s -aa-eval -print-all-alias-modref-info -disable-output \ +; RUN: |& grep {NoAlias: double\\* \[%\]a, double\\* \[%\]b\$} | count 4 + +; BasicAA should detect NoAliases in PHIs and Selects. + +; Two PHIs in the same block. +define void @foo(i1 %m, double* noalias %x, double* noalias %y) { +entry: + br i1 %m, label %true, label %false + +true: + br label %exit + +false: + br label %exit + +exit: + %a = phi double* [ %x, %true ], [ %y, %false ] + %b = phi double* [ %x, %false ], [ %y, %true ] + volatile store double 0.0, double* %a + volatile store double 1.0, double* %b + ret void +} + +; Two selects with the same condition. +define void @bar(i1 %m, double* noalias %x, double* noalias %y) { +entry: + %a = select i1 %m, double* %x, double* %y + %b = select i1 %m, double* %y, double* %x + volatile store double 0.000000e+00, double* %a + volatile store double 1.000000e+00, double* %b + ret void +} + +; Two PHIs with disjoint sets of inputs. +define void @qux(i1 %m, double* noalias %x, double* noalias %y, + i1 %n, double* noalias %v, double* noalias %w) { +entry: + br i1 %m, label %true, label %false + +true: + br label %exit + +false: + br label %exit + +exit: + %a = phi double* [ %x, %true ], [ %y, %false ] + br i1 %n, label %ntrue, label %nfalse + +ntrue: + br label %nexit + +nfalse: + br label %nexit + +nexit: + %b = phi double* [ %v, %ntrue ], [ %w, %nfalse ] + volatile store double 0.0, double* %a + volatile store double 1.0, double* %b + ret void +} + +; Two selects with disjoint sets of arms. +define void @fin(i1 %m, double* noalias %x, double* noalias %y, + i1 %n, double* noalias %v, double* noalias %w) { +entry: + %a = select i1 %m, double* %x, double* %y + %b = select i1 %n, double* %v, double* %w + volatile store double 0.000000e+00, double* %a + volatile store double 1.000000e+00, double* %b + ret void +} From kremenek at apple.com Mon Oct 26 17:06:01 2009 From: kremenek at apple.com (Ted Kremenek) Date: Mon, 26 Oct 2009 22:06:01 -0000 Subject: [llvm-commits] [llvm] r85161 - in /llvm/trunk/lib: Target/MSP430/AsmPrinter/CMakeLists.txt Transforms/Scalar/CMakeLists.txt Message-ID: <200910262206.n9QM61eR006887@zion.cs.uiuc.edu> Author: kremenek Date: Mon Oct 26 17:06:01 2009 New Revision: 85161 URL: http://llvm.org/viewvc/llvm-project?rev=85161&view=rev Log: Update CMake files. Modified: llvm/trunk/lib/Target/MSP430/AsmPrinter/CMakeLists.txt llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt Modified: llvm/trunk/lib/Target/MSP430/AsmPrinter/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/MSP430/AsmPrinter/CMakeLists.txt?rev=85161&r1=85160&r2=85161&view=diff ============================================================================== --- llvm/trunk/lib/Target/MSP430/AsmPrinter/CMakeLists.txt (original) +++ llvm/trunk/lib/Target/MSP430/AsmPrinter/CMakeLists.txt Mon Oct 26 17:06:01 2009 @@ -1,8 +1,8 @@ include_directories( ${CMAKE_CURRENT_BINARY_DIR}/.. ${CMAKE_CURRENT_SOURCE_DIR}/.. ) add_llvm_library(LLVMMSP430AsmPrinter - MSP430InstPrinter.cpp MSP430AsmPrinter.cpp + MSP430InstPrinter.cpp MSP430MCInstLower.cpp ) add_dependencies(LLVMMSP430AsmPrinter MSP430CodeGenTable_gen) Modified: llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt?rev=85161&r1=85160&r2=85161&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt (original) +++ llvm/trunk/lib/Transforms/Scalar/CMakeLists.txt Mon Oct 26 17:06:01 2009 @@ -7,6 +7,7 @@ ConstantProp.cpp DCE.cpp DeadStoreElimination.cpp + GEPSplitter.cpp GVN.cpp IndVarSimplify.cpp InstructionCombining.cpp From gohman at apple.com Mon Oct 26 17:14:23 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 22:14:23 -0000 Subject: [llvm-commits] [llvm] r85164 - in /llvm/trunk: lib/Transforms/Scalar/InstructionCombining.cpp test/Transforms/InstCombine/add-shrink.ll test/Transforms/InstCombine/add-sitofp.ll Message-ID: <200910262214.n9QMENxO007270@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 17:14:22 2009 New Revision: 85164 URL: http://llvm.org/viewvc/llvm-project?rev=85164&view=rev Log: Code that checks WillNotOverflowSignedAdd before creating an Add can safely use the NSW bit on the Add. Modified: llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp llvm/trunk/test/Transforms/InstCombine/add-shrink.ll llvm/trunk/test/Transforms/InstCombine/add-sitofp.ll Modified: llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp?rev=85164&r1=85163&r2=85164&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp (original) +++ llvm/trunk/lib/Transforms/Scalar/InstructionCombining.cpp Mon Oct 26 17:14:22 2009 @@ -2420,8 +2420,8 @@ ConstantExpr::getSExt(CI, I.getType()) == RHSC && WillNotOverflowSignedAdd(LHSConv->getOperand(0), CI)) { // Insert the new, smaller add. - Value *NewAdd = Builder->CreateAdd(LHSConv->getOperand(0), - CI, "addconv"); + Value *NewAdd = Builder->CreateNSWAdd(LHSConv->getOperand(0), + CI, "addconv"); return new SExtInst(NewAdd, I.getType()); } } @@ -2436,8 +2436,8 @@ WillNotOverflowSignedAdd(LHSConv->getOperand(0), RHSConv->getOperand(0))) { // Insert the new integer add. - Value *NewAdd = Builder->CreateAdd(LHSConv->getOperand(0), - RHSConv->getOperand(0), "addconv"); + Value *NewAdd = Builder->CreateNSWAdd(LHSConv->getOperand(0), + RHSConv->getOperand(0), "addconv"); return new SExtInst(NewAdd, I.getType()); } } @@ -2493,8 +2493,8 @@ ConstantExpr::getSIToFP(CI, I.getType()) == CFP && WillNotOverflowSignedAdd(LHSConv->getOperand(0), CI)) { // Insert the new integer add. - Value *NewAdd = Builder->CreateAdd(LHSConv->getOperand(0), - CI, "addconv"); + Value *NewAdd = Builder->CreateNSWAdd(LHSConv->getOperand(0), + CI, "addconv"); return new SIToFPInst(NewAdd, I.getType()); } } @@ -2509,8 +2509,8 @@ WillNotOverflowSignedAdd(LHSConv->getOperand(0), RHSConv->getOperand(0))) { // Insert the new integer add. - Value *NewAdd = Builder->CreateAdd(LHSConv->getOperand(0), - RHSConv->getOperand(0), "addconv"); + Value *NewAdd = Builder->CreateNSWAdd(LHSConv->getOperand(0), + RHSConv->getOperand(0), "addconv"); return new SIToFPInst(NewAdd, I.getType()); } } Modified: llvm/trunk/test/Transforms/InstCombine/add-shrink.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/InstCombine/add-shrink.ll?rev=85164&r1=85163&r2=85164&view=diff ============================================================================== --- llvm/trunk/test/Transforms/InstCombine/add-shrink.ll (original) +++ llvm/trunk/test/Transforms/InstCombine/add-shrink.ll Mon Oct 26 17:14:22 2009 @@ -1,4 +1,4 @@ -; RUN: opt < %s -instcombine -S | grep {add i32} +; RUN: opt < %s -instcombine -S | grep {add nsw i32} ; RUN: opt < %s -instcombine -S | grep sext | count 1 ; Should only have one sext and the add should be i32 instead of i64. Modified: llvm/trunk/test/Transforms/InstCombine/add-sitofp.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/InstCombine/add-sitofp.ll?rev=85164&r1=85163&r2=85164&view=diff ============================================================================== --- llvm/trunk/test/Transforms/InstCombine/add-sitofp.ll (original) +++ llvm/trunk/test/Transforms/InstCombine/add-sitofp.ll Mon Oct 26 17:14:22 2009 @@ -1,4 +1,4 @@ -; RUN: opt < %s -instcombine -S | grep {add i32} +; RUN: opt < %s -instcombine -S | grep {add nsw i32} define double @x(i32 %a, i32 %b) nounwind { %m = lshr i32 %a, 24 From gohman at apple.com Mon Oct 26 17:18:58 2009 From: gohman at apple.com (Dan Gohman) Date: Mon, 26 Oct 2009 22:18:58 -0000 Subject: [llvm-commits] [llvm] r85165 - /llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp Message-ID: <200910262218.n9QMIxVo007520@zion.cs.uiuc.edu> Author: djg Date: Mon Oct 26 17:18:58 2009 New Revision: 85165 URL: http://llvm.org/viewvc/llvm-project?rev=85165&view=rev Log: Simplify this code. LoopDeletion doesn't need to explicit check that the loop exiting block dominates the latch block; if ScalarEvolution can prove that the trip-count is finite, that's sufficient. Modified: llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp Modified: llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp?rev=85165&r1=85164&r2=85165&view=diff ============================================================================== --- llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp (original) +++ llvm/trunk/lib/Transforms/Scalar/LoopDeletion.cpp Mon Oct 26 17:18:58 2009 @@ -33,8 +33,6 @@ // Possibly eliminate loop L if it is dead. bool runOnLoop(Loop* L, LPPassManager& LPM); - bool SingleDominatingExit(Loop* L, - SmallVector& exitingBlocks); bool IsLoopDead(Loop* L, SmallVector& exitingBlocks, SmallVector& exitBlocks, bool &Changed, BasicBlock *Preheader); @@ -63,25 +61,6 @@ return new LoopDeletion(); } -/// SingleDominatingExit - Checks that there is only a single blocks that -/// branches out of the loop, and that it also g the latch block. Loops -/// with multiple or non-latch-dominating exiting blocks could be dead, but we'd -/// have to do more extensive analysis to make sure, for instance, that the -/// control flow logic involved was or could be made loop-invariant. -bool LoopDeletion::SingleDominatingExit(Loop* L, - SmallVector& exitingBlocks) { - - if (exitingBlocks.size() != 1) - return false; - - BasicBlock* latch = L->getLoopLatch(); - if (!latch) - return false; - - DominatorTree& DT = getAnalysis(); - return DT.dominates(exitingBlocks[0], latch); -} - /// IsLoopDead - Determined if a loop is dead. This assumes that we've already /// checked for unique exit and exiting blocks, and that the code is in LCSSA /// form. @@ -154,9 +133,8 @@ if (exitBlocks.size() != 1) return false; - // Loops with multiple exits or exits that don't dominate the latch - // are too complicated to handle correctly. - if (!SingleDominatingExit(L, exitingBlocks)) + // Loops with multiple exits are too complicated to handle correctly. + if (exitingBlocks.size() != 1) return false; // Finally, we have to check that the loop really is dead. From david_goodwin at apple.com Mon Oct 26 17:31:19 2009 From: david_goodwin at apple.com (David Goodwin) Date: Mon, 26 Oct 2009 22:31:19 -0000 Subject: [llvm-commits] [llvm] r85166 - in /llvm/trunk: include/llvm/CodeGen/AntiDepBreaker.h lib/CodeGen/AggressiveAntiDepBreaker.cpp lib/CodeGen/AggressiveAntiDepBreaker.h lib/CodeGen/CriticalAntiDepBreaker.h lib/CodeGen/PostRASchedulerList.cpp Message-ID: <200910262231.n9QMVJwG008113@zion.cs.uiuc.edu> Author: david_goodwin Date: Mon Oct 26 17:31:16 2009 New Revision: 85166 URL: http://llvm.org/viewvc/llvm-project?rev=85166&view=rev Log: Allow the aggressive anti-dep breaker to process the same region multiple times. This is necessary because new anti-dependencies are exposed when "current" ones are broken. Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Modified: llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h?rev=85166&r1=85165&r2=85166&view=diff ============================================================================== --- llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h (original) +++ llvm/trunk/include/llvm/CodeGen/AntiDepBreaker.h Mon Oct 26 17:31:16 2009 @@ -31,6 +31,10 @@ public: virtual ~AntiDepBreaker(); + /// GetMaxTrials - Return the maximum number of anti-dependence + /// breaking attempts that will be made for a block. + virtual unsigned GetMaxTrials() =0; + /// Start - Initialize anti-dep breaking for a new basic block. virtual void StartBlock(MachineBasicBlock *BB) =0; Modified: llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp?rev=85166&r1=85165&r2=85166&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp (original) +++ llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.cpp Mon Oct 26 17:31:16 2009 @@ -22,36 +22,109 @@ #include "llvm/Target/TargetInstrInfo.h" #include "llvm/Target/TargetMachine.h" #include "llvm/Target/TargetRegisterInfo.h" +#include "llvm/Support/CommandLine.h" #include "llvm/Support/Debug.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/raw_ostream.h" - using namespace llvm; +static cl::opt +AntiDepTrials("agg-antidep-trials", + cl::desc("Maximum number of anti-dependency breaking passes"), + cl::init(2), cl::Hidden); + +AggressiveAntiDepState::AggressiveAntiDepState(MachineBasicBlock *BB) : + GroupNodes(TargetRegisterInfo::FirstVirtualRegister, 0) { + // Initialize all registers to be in their own group. Initially we + // assign the register to the same-indexed GroupNode. + for (unsigned i = 0; i < TargetRegisterInfo::FirstVirtualRegister; ++i) + GroupNodeIndices[i] = i; + + // Initialize the indices to indicate that no registers are live. + std::fill(KillIndices, array_endof(KillIndices), ~0u); + std::fill(DefIndices, array_endof(DefIndices), BB->size()); +} + +unsigned AggressiveAntiDepState::GetGroup(unsigned Reg) +{ + unsigned Node = GroupNodeIndices[Reg]; + while (GroupNodes[Node] != Node) + Node = GroupNodes[Node]; + + return Node; +} + +void AggressiveAntiDepState::GetGroupRegs(unsigned Group, std::vector &Regs) +{ + for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) { + if (GetGroup(Reg) == Group) + Regs.push_back(Reg); + } +} + +unsigned AggressiveAntiDepState::UnionGroups(unsigned Reg1, unsigned Reg2) +{ + assert(GroupNodes[0] == 0 && "GroupNode 0 not parent!"); + assert(GroupNodeIndices[0] == 0 && "Reg 0 not in Group 0!"); + + // find group for each register + unsigned Group1 = GetGroup(Reg1); + unsigned Group2 = GetGroup(Reg2); + + // if either group is 0, then that must become the parent + unsigned Parent = (Group1 == 0) ? Group1 : Group2; + unsigned Other = (Parent == Group1) ? Group2 : Group1; + GroupNodes.at(Other) = Parent; + return Parent; +} + +unsigned AggressiveAntiDepState::LeaveGroup(unsigned Reg) +{ + // Create a new GroupNode for Reg. Reg's existing GroupNode must + // stay as is because there could be other GroupNodes referring to + // it. + unsigned idx = GroupNodes.size(); + GroupNodes.push_back(idx); + GroupNodeIndices[Reg] = idx; + return idx; +} + +bool AggressiveAntiDepState::IsLive(unsigned Reg) +{ + // KillIndex must be defined and DefIndex not defined for a register + // to be live. + return((KillIndices[Reg] != ~0u) && (DefIndices[Reg] == ~0u)); +} + + + AggressiveAntiDepBreaker:: AggressiveAntiDepBreaker(MachineFunction& MFi) : AntiDepBreaker(), MF(MFi), MRI(MF.getRegInfo()), TRI(MF.getTarget().getRegisterInfo()), AllocatableSet(TRI->getAllocatableSet(MF)), - GroupNodes(TargetRegisterInfo::FirstVirtualRegister, 0) -{ + State(NULL), SavedState(NULL) { } AggressiveAntiDepBreaker::~AggressiveAntiDepBreaker() { + delete State; + delete SavedState; } -void AggressiveAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { - // Initialize all registers to be in their own group. Initially we - // assign the register to the same-indexed GroupNode. - for (unsigned i = 0; i < TargetRegisterInfo::FirstVirtualRegister; ++i) - GroupNodeIndices[i] = i; +unsigned AggressiveAntiDepBreaker::GetMaxTrials() { + if (AntiDepTrials <= 0) + return 1; + return AntiDepTrials; +} - // Initialize the indices to indicate that no registers are live. - std::fill(KillIndices, array_endof(KillIndices), ~0u); - std::fill(DefIndices, array_endof(DefIndices), BB->size()); +void AggressiveAntiDepBreaker::StartBlock(MachineBasicBlock *BB) { + assert(State == NULL); + State = new AggressiveAntiDepState(BB); bool IsReturnBlock = (!BB->empty() && BB->back().getDesc().isReturn()); + unsigned *KillIndices = State->GetKillIndices(); + unsigned *DefIndices = State->GetDefIndices(); // Determine the live-out physregs for this block. if (IsReturnBlock) { @@ -59,13 +132,13 @@ for (MachineRegisterInfo::liveout_iterator I = MRI.liveout_begin(), E = MRI.liveout_end(); I != E; ++I) { unsigned Reg = *I; - UnionGroups(Reg, 0); + State->UnionGroups(Reg, 0); KillIndices[Reg] = BB->size(); DefIndices[Reg] = ~0u; // Repeat, for all aliases. for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { unsigned AliasReg = *Alias; - UnionGroups(AliasReg, 0); + State->UnionGroups(AliasReg, 0); KillIndices[AliasReg] = BB->size(); DefIndices[AliasReg] = ~0u; } @@ -77,13 +150,13 @@ for (MachineBasicBlock::livein_iterator I = (*SI)->livein_begin(), E = (*SI)->livein_end(); I != E; ++I) { unsigned Reg = *I; - UnionGroups(Reg, 0); + State->UnionGroups(Reg, 0); KillIndices[Reg] = BB->size(); DefIndices[Reg] = ~0u; // Repeat, for all aliases. for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { unsigned AliasReg = *Alias; - UnionGroups(AliasReg, 0); + State->UnionGroups(AliasReg, 0); KillIndices[AliasReg] = BB->size(); DefIndices[AliasReg] = ~0u; } @@ -98,13 +171,13 @@ for (const unsigned *I = TRI->getCalleeSavedRegs(); *I; ++I) { unsigned Reg = *I; if (!IsReturnBlock && !Pristine.test(Reg)) continue; - UnionGroups(Reg, 0); + State->UnionGroups(Reg, 0); KillIndices[Reg] = BB->size(); DefIndices[Reg] = ~0u; // Repeat, for all aliases. for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { unsigned AliasReg = *Alias; - UnionGroups(AliasReg, 0); + State->UnionGroups(AliasReg, 0); KillIndices[AliasReg] = BB->size(); DefIndices[AliasReg] = ~0u; } @@ -112,7 +185,10 @@ } void AggressiveAntiDepBreaker::FinishBlock() { - RegRefs.clear(); + delete State; + State = NULL; + delete SavedState; + SavedState = NULL; } void AggressiveAntiDepBreaker::Observe(MachineInstr *MI, unsigned Count, @@ -122,6 +198,7 @@ DEBUG(errs() << "Observe: "); DEBUG(MI->dump()); + unsigned *DefIndices = State->GetDefIndices(); for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) { // If Reg is current live, then mark that it can't be renamed as // we don't know the extent of its live-range anymore (now that it @@ -129,11 +206,11 @@ // previous schedule region, then set its def index to the most // conservative location (i.e. the beginning of the previous // schedule region). - if (IsLive(Reg)) { - DEBUG(if (GetGroup(Reg) != 0) + if (State->IsLive(Reg)) { + DEBUG(if (State->GetGroup(Reg) != 0) errs() << " " << TRI->getName(Reg) << "=g" << - GetGroup(Reg) << "->g0(region live-out)"); - UnionGroups(Reg, 0); + State->GetGroup(Reg) << "->g0(region live-out)"); + State->UnionGroups(Reg, 0); } else if ((DefIndices[Reg] < InsertPosIndex) && (DefIndices[Reg] >= Count)) { DefIndices[Reg] = Count; } @@ -143,57 +220,10 @@ GetPassthruRegs(MI, PassthruRegs); PrescanInstruction(MI, Count, PassthruRegs); ScanInstruction(MI, Count); -} - -unsigned AggressiveAntiDepBreaker::GetGroup(unsigned Reg) -{ - unsigned Node = GroupNodeIndices[Reg]; - while (GroupNodes[Node] != Node) - Node = GroupNodes[Node]; - return Node; -} - -void AggressiveAntiDepBreaker::GetGroupRegs(unsigned Group, std::vector &Regs) -{ - for (unsigned Reg = 0; Reg != TargetRegisterInfo::FirstVirtualRegister; ++Reg) { - if (GetGroup(Reg) == Group) - Regs.push_back(Reg); - } -} - -unsigned AggressiveAntiDepBreaker::UnionGroups(unsigned Reg1, unsigned Reg2) -{ - assert(GroupNodes[0] == 0 && "GroupNode 0 not parent!"); - assert(GroupNodeIndices[0] == 0 && "Reg 0 not in Group 0!"); - - // find group for each register - unsigned Group1 = GetGroup(Reg1); - unsigned Group2 = GetGroup(Reg2); - - // if either group is 0, then that must become the parent - unsigned Parent = (Group1 == 0) ? Group1 : Group2; - unsigned Other = (Parent == Group1) ? Group2 : Group1; - GroupNodes.at(Other) = Parent; - return Parent; -} - -unsigned AggressiveAntiDepBreaker::LeaveGroup(unsigned Reg) -{ - // Create a new GroupNode for Reg. Reg's existing GroupNode must - // stay as is because there could be other GroupNodes referring to - // it. - unsigned idx = GroupNodes.size(); - GroupNodes.push_back(idx); - GroupNodeIndices[Reg] = idx; - return idx; -} - -bool AggressiveAntiDepBreaker::IsLive(unsigned Reg) -{ - // KillIndex must be defined and DefIndex not defined for a register - // to be live. - return((KillIndices[Reg] != ~0u) && (DefIndices[Reg] == ~0u)); + // We're starting a new schedule region so forget any saved state. + delete SavedState; + SavedState = NULL; } bool AggressiveAntiDepBreaker::IsImplicitDefUse(MachineInstr *MI, @@ -249,6 +279,10 @@ void AggressiveAntiDepBreaker::PrescanInstruction(MachineInstr *MI, unsigned Count, std::set& PassthruRegs) { + unsigned *DefIndices = State->GetDefIndices(); + std::multimap& + RegRefs = State->GetRegRefs(); + // Scan the register defs for this instruction and update // live-ranges, groups and RegRefs. for (unsigned i = 0, e = MI->getNumOperands(); i != e; ++i) { @@ -275,23 +309,23 @@ unsigned Reg = MO.getReg(); if (Reg == 0) continue; - DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << GetGroup(Reg)); + DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << State->GetGroup(Reg)); // If MI's defs have special allocation requirement, don't allow // any def registers to be changed. Also assume all registers // defined in a call must not be changed (ABI). if (MI->getDesc().isCall() || MI->getDesc().hasExtraDefRegAllocReq()) { - DEBUG(if (GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); - UnionGroups(Reg, 0); + DEBUG(if (State->GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); + State->UnionGroups(Reg, 0); } // Any aliased that are live at this point are completely or // partially defined here, so group those subregisters with Reg. for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias) { unsigned AliasReg = *Alias; - if (IsLive(AliasReg)) { - UnionGroups(Reg, AliasReg); - DEBUG(errs() << "->g" << GetGroup(Reg) << "(via " << + if (State->IsLive(AliasReg)) { + State->UnionGroups(Reg, AliasReg); + DEBUG(errs() << "->g" << State->GetGroup(Reg) << "(via " << TRI->getName(AliasReg) << ")"); } } @@ -300,7 +334,7 @@ const TargetRegisterClass *RC = NULL; if (i < MI->getDesc().getNumOperands()) RC = MI->getDesc().OpInfo[i].getRegClass(TRI); - RegisterReference RR = { &MO, RC }; + AggressiveAntiDepState::RegisterReference RR = { &MO, RC }; RegRefs.insert(std::make_pair(Reg, RR)); } @@ -310,6 +344,10 @@ void AggressiveAntiDepBreaker::ScanInstruction(MachineInstr *MI, unsigned Count) { DEBUG(errs() << "\tUse Groups:"); + unsigned *KillIndices = State->GetKillIndices(); + unsigned *DefIndices = State->GetDefIndices(); + std::multimap& + RegRefs = State->GetRegRefs(); // Scan the register uses for this instruction and update // live-ranges, groups and RegRefs. @@ -319,29 +357,30 @@ unsigned Reg = MO.getReg(); if (Reg == 0) continue; - DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << GetGroup(Reg)); + DEBUG(errs() << " " << TRI->getName(Reg) << "=g" << + State->GetGroup(Reg)); // It wasn't previously live but now it is, this is a kill. Forget // the previous live-range information and start a new live-range // for the register. - if (!IsLive(Reg)) { + if (!State->IsLive(Reg)) { KillIndices[Reg] = Count; DefIndices[Reg] = ~0u; RegRefs.erase(Reg); - LeaveGroup(Reg); - DEBUG(errs() << "->g" << GetGroup(Reg) << "(last-use)"); + State->LeaveGroup(Reg); + DEBUG(errs() << "->g" << State->GetGroup(Reg) << "(last-use)"); } // Repeat, for subregisters. for (const unsigned *Subreg = TRI->getSubRegisters(Reg); *Subreg; ++Subreg) { unsigned SubregReg = *Subreg; - if (!IsLive(SubregReg)) { + if (!State->IsLive(SubregReg)) { KillIndices[SubregReg] = Count; DefIndices[SubregReg] = ~0u; RegRefs.erase(SubregReg); - LeaveGroup(SubregReg); + State->LeaveGroup(SubregReg); DEBUG(errs() << " " << TRI->getName(SubregReg) << "->g" << - GetGroup(SubregReg) << "(last-use)"); + State->GetGroup(SubregReg) << "(last-use)"); } } @@ -349,15 +388,15 @@ // any use registers to be changed. Also assume all registers // used in a call must not be changed (ABI). if (MI->getDesc().isCall() || MI->getDesc().hasExtraSrcRegAllocReq()) { - DEBUG(if (GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); - UnionGroups(Reg, 0); + DEBUG(if (State->GetGroup(Reg) != 0) errs() << "->g0(alloc-req)"); + State->UnionGroups(Reg, 0); } // Note register reference... const TargetRegisterClass *RC = NULL; if (i < MI->getDesc().getNumOperands()) RC = MI->getDesc().OpInfo[i].getRegClass(TRI); - RegisterReference RR = { &MO, RC }; + AggressiveAntiDepState::RegisterReference RR = { &MO, RC }; RegRefs.insert(std::make_pair(Reg, RR)); } @@ -377,14 +416,14 @@ if (FirstReg != 0) { DEBUG(errs() << "=" << TRI->getName(Reg)); - UnionGroups(FirstReg, Reg); + State->UnionGroups(FirstReg, Reg); } else { DEBUG(errs() << " " << TRI->getName(Reg)); FirstReg = Reg; } } - DEBUG(errs() << "->g" << GetGroup(FirstReg) << '\n'); + DEBUG(errs() << "->g" << State->GetGroup(FirstReg) << '\n'); } } @@ -395,10 +434,12 @@ // Check all references that need rewriting for Reg. For each, use // the corresponding register class to narrow the set of registers // that are appropriate for renaming. - std::pair::iterator, - std::multimap::iterator> - Range = RegRefs.equal_range(Reg); - for (std::multimap::iterator + std::pair::iterator, + std::multimap::iterator> + Range = State->GetRegRefs().equal_range(Reg); + for (std::multimap::iterator Q = Range.first, QE = Range.second; Q != QE; ++Q) { const TargetRegisterClass *RC = Q->second.RC; if (RC == NULL) continue; @@ -420,11 +461,16 @@ bool AggressiveAntiDepBreaker::FindSuitableFreeRegisters( unsigned AntiDepGroupIndex, std::map &RenameMap) { + unsigned *KillIndices = State->GetKillIndices(); + unsigned *DefIndices = State->GetDefIndices(); + std::multimap& + RegRefs = State->GetRegRefs(); + // Collect all registers in the same group as AntiDepReg. These all // need to be renamed together if we are to break the // anti-dependence. std::vector Regs; - GetGroupRegs(AntiDepGroupIndex, Regs); + State->GetGroupRegs(AntiDepGroupIndex, Regs); assert(Regs.size() > 0 && "Empty register group!"); if (Regs.size() == 0) return false; @@ -484,7 +530,7 @@ // If Reg is dead and Reg's most recent def is not before // SuperRegs's kill, it's safe to replace SuperReg with // Reg. We must also check all subregisters of Reg. - if (IsLive(Reg) || (KillIndices[SuperReg] > DefIndices[Reg])) { + if (State->IsLive(Reg) || (KillIndices[SuperReg] > DefIndices[Reg])) { DEBUG(errs() << "(live)"); continue; } else { @@ -492,7 +538,7 @@ for (const unsigned *Subreg = TRI->getSubRegisters(Reg); *Subreg; ++Subreg) { unsigned SubregReg = *Subreg; - if (IsLive(SubregReg) || (KillIndices[SuperReg] > DefIndices[SubregReg])) { + if (State->IsLive(SubregReg) || (KillIndices[SuperReg] > DefIndices[SubregReg])) { DEBUG(errs() << "(subreg " << TRI->getName(SubregReg) << " live)"); found = true; break; @@ -518,13 +564,29 @@ /// BreakAntiDependencies - Identifiy anti-dependencies within the /// ScheduleDAG and break them by renaming registers. /// -unsigned AggressiveAntiDepBreaker::BreakAntiDependencies(std::vector& SUnits, - MachineBasicBlock::iterator& Begin, - MachineBasicBlock::iterator& End, - unsigned InsertPosIndex) { +unsigned AggressiveAntiDepBreaker::BreakAntiDependencies( + std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex) { + unsigned *KillIndices = State->GetKillIndices(); + unsigned *DefIndices = State->GetDefIndices(); + std::multimap& + RegRefs = State->GetRegRefs(); + // The code below assumes that there is at least one instruction, // so just duck out immediately if the block is empty. if (SUnits.empty()) return false; + + // Manage saved state to enable multiple passes... + if (AntiDepTrials > 1) { + if (SavedState == NULL) { + SavedState = new AggressiveAntiDepState(*State); + } else { + delete State; + State = new AggressiveAntiDepState(*SavedState); + } + } // ...need a map from MI to SUnit. std::map MISUnitMap; @@ -539,7 +601,7 @@ { DEBUG(errs() << "Available regs:"); for (unsigned Reg = 0; Reg < TRI->getNumRegs(); ++Reg) { - if (!IsLive(Reg)) + if (!State->IsLive(Reg)) DEBUG(errs() << " " << TRI->getName(Reg)); } DEBUG(errs() << '\n'); @@ -623,7 +685,7 @@ if (AntiDepReg == 0) continue; // Determine AntiDepReg's register group. - const unsigned GroupIndex = GetGroup(AntiDepReg); + const unsigned GroupIndex = State->GetGroup(AntiDepReg); if (GroupIndex == 0) { DEBUG(errs() << " (zero group)\n"); continue; @@ -649,10 +711,12 @@ // Update the references to the old register CurrReg to // refer to the new register NewReg. - std::pair::iterator, - std::multimap::iterator> + std::pair::iterator, + std::multimap::iterator> Range = RegRefs.equal_range(CurrReg); - for (std::multimap::iterator + for (std::multimap::iterator Q = Range.first, QE = Range.second; Q != QE; ++Q) { Q->second.Operand->setReg(NewReg); } @@ -660,12 +724,12 @@ // We just went back in time and modified history; the // liveness information for CurrReg is now inconsistent. Set // the state as if it were dead. - UnionGroups(NewReg, 0); + State->UnionGroups(NewReg, 0); RegRefs.erase(NewReg); DefIndices[NewReg] = DefIndices[CurrReg]; KillIndices[NewReg] = KillIndices[CurrReg]; - UnionGroups(CurrReg, 0); + State->UnionGroups(CurrReg, 0); RegRefs.erase(CurrReg); DefIndices[CurrReg] = KillIndices[CurrReg]; KillIndices[CurrReg] = ~0u; Modified: llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h?rev=85166&r1=85165&r2=85166&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h (original) +++ llvm/trunk/lib/CodeGen/AggressiveAntiDepBreaker.h Mon Oct 26 17:31:16 2009 @@ -28,11 +28,12 @@ #include "llvm/ADT/SmallSet.h" namespace llvm { - class AggressiveAntiDepBreaker : public AntiDepBreaker { - MachineFunction& MF; - MachineRegisterInfo &MRI; - const TargetRegisterInfo *TRI; - + /// Class AggressiveAntiDepState + /// Contains all the state necessary for anti-dep breaking. We place + /// into a separate class so be can conveniently save/restore it to + /// enable multi-pass anti-dep breaking. + class AggressiveAntiDepState { + public: /// RegisterReference - Information about a register reference /// within a liverange typedef struct { @@ -42,59 +43,43 @@ const TargetRegisterClass *RC; } RegisterReference; - /// AllocatableSet - The set of allocatable registers. - /// We'll be ignoring anti-dependencies on non-allocatable registers, - /// because they may not be safe to break. - const BitVector AllocatableSet; - + private: /// GroupNodes - Implements a disjoint-union data structure to /// form register groups. A node is represented by an index into /// the vector. A node can "point to" itself to indicate that it /// is the parent of a group, or point to another node to indicate /// that it is a member of the same group as that node. std::vector GroupNodes; - + /// GroupNodeIndices - For each register, the index of the GroupNode /// currently representing the group that the register belongs to. /// Register 0 is always represented by the 0 group, a group /// composed of registers that are not eligible for anti-aliasing. unsigned GroupNodeIndices[TargetRegisterInfo::FirstVirtualRegister]; - - /// RegRegs - Map registers to all their references within a live range. + + /// RegRefs - Map registers to all their references within a live range. std::multimap RegRefs; - + /// KillIndices - The index of the most recent kill (proceding bottom-up), /// or ~0u if the register is not live. unsigned KillIndices[TargetRegisterInfo::FirstVirtualRegister]; - + /// DefIndices - The index of the most recent complete def (proceding bottom /// up), or ~0u if the register is live. unsigned DefIndices[TargetRegisterInfo::FirstVirtualRegister]; public: - AggressiveAntiDepBreaker(MachineFunction& MFi); - ~AggressiveAntiDepBreaker(); + AggressiveAntiDepState(MachineBasicBlock *BB); - /// Start - Initialize anti-dep breaking for a new basic block. - void StartBlock(MachineBasicBlock *BB); - - /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path - /// of the ScheduleDAG and break them by renaming registers. - /// - unsigned BreakAntiDependencies(std::vector& SUnits, - MachineBasicBlock::iterator& Begin, - MachineBasicBlock::iterator& End, - unsigned InsertPosIndex); + /// GetKillIndices - Return the kill indices. + unsigned *GetKillIndices() { return KillIndices; } - /// Observe - Update liveness information to account for the current - /// instruction, which will not be scheduled. - /// - void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); + /// GetDefIndices - Return the define indices. + unsigned *GetDefIndices() { return DefIndices; } - /// Finish - Finish anti-dep breaking for a basic block. - void FinishBlock(); + /// GetRegRefs - Return the RegRefs map. + std::multimap& GetRegRefs() { return RegRefs; } - private: // GetGroup - Get the group for a register. The returned value is // the index of the GroupNode representing the group. unsigned GetGroup(unsigned Reg); @@ -115,7 +100,57 @@ /// IsLive - Return true if Reg is live bool IsLive(unsigned Reg); + }; + + + /// Class AggressiveAntiDepBreaker + class AggressiveAntiDepBreaker : public AntiDepBreaker { + MachineFunction& MF; + MachineRegisterInfo &MRI; + const TargetRegisterInfo *TRI; + + /// AllocatableSet - The set of allocatable registers. + /// We'll be ignoring anti-dependencies on non-allocatable registers, + /// because they may not be safe to break. + const BitVector AllocatableSet; + + /// State - The state used to identify and rename anti-dependence + /// registers. + AggressiveAntiDepState *State; + + /// SavedState - The state for the start of an anti-dep + /// region. Used to restore the state at the beginning of each + /// pass + AggressiveAntiDepState *SavedState; + + public: + AggressiveAntiDepBreaker(MachineFunction& MFi); + ~AggressiveAntiDepBreaker(); + /// GetMaxTrials - As anti-dependencies are broken, additional + /// dependencies may be exposed, so multiple passes are required. + unsigned GetMaxTrials(); + + /// Start - Initialize anti-dep breaking for a new basic block. + void StartBlock(MachineBasicBlock *BB); + + /// BreakAntiDependencies - Identifiy anti-dependencies along the critical path + /// of the ScheduleDAG and break them by renaming registers. + /// + unsigned BreakAntiDependencies(std::vector& SUnits, + MachineBasicBlock::iterator& Begin, + MachineBasicBlock::iterator& End, + unsigned InsertPosIndex); + + /// Observe - Update liveness information to account for the current + /// instruction, which will not be scheduled. + /// + void Observe(MachineInstr *MI, unsigned Count, unsigned InsertPosIndex); + + /// Finish - Finish anti-dep breaking for a basic block. + void FinishBlock(); + + private: /// IsImplicitDefUse - Return true if MO represents a register /// that is both implicitly used and defined in MI bool IsImplicitDefUse(MachineInstr *MI, MachineOperand& MO); Modified: llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h?rev=85166&r1=85165&r2=85166&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h (original) +++ llvm/trunk/lib/CodeGen/CriticalAntiDepBreaker.h Mon Oct 26 17:31:16 2009 @@ -64,6 +64,10 @@ CriticalAntiDepBreaker(MachineFunction& MFi); ~CriticalAntiDepBreaker(); + /// GetMaxTrials - Critical path anti-dependence breaking requires + /// only a single pass + unsigned GetMaxTrials() { return 1; } + /// Start - Initialize anti-dep breaking for a new basic block. void StartBlock(MachineBasicBlock *BB); Modified: llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp?rev=85166&r1=85165&r2=85166&view=diff ============================================================================== --- llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp (original) +++ llvm/trunk/lib/CodeGen/PostRASchedulerList.cpp Mon Oct 26 17:31:16 2009 @@ -40,6 +40,7 @@ #include "llvm/Target/TargetInstrInfo.h" #include "llvm/Target/TargetRegisterInfo.h" #include "llvm/Target/TargetSubtarget.h" +#include "llvm/Support/CommandLine.h" #include "llvm/Support/Debug.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/raw_ostream.h" @@ -315,16 +316,20 @@ /// Schedule - Schedule the instruction range using list scheduling. /// void SchedulePostRATDList::Schedule() { - DEBUG(errs() << "********** List Scheduling **********\n"); - // Build the scheduling graph. BuildSchedGraph(AA); if (AntiDepBreak != NULL) { - unsigned Broken = - AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, - InsertPosIndex); - if (Broken > 0) { + for (unsigned i = 0, Trials = AntiDepBreak->GetMaxTrials(); + i < Trials; ++i) { + DEBUG(errs() << "********** Break Anti-Deps, Trial " << + i << " **********\n"); + unsigned Broken = + AntiDepBreak->BreakAntiDependencies(SUnits, Begin, InsertPos, + InsertPosIndex); + if (Broken == 0) + break; + // We made changes. Update the dependency graph. // Theoretically we could update the graph in place: // When a live range is changed to use a different register, remove @@ -340,6 +345,8 @@ } } + DEBUG(errs() << "********** List Scheduling **********\n"); + DEBUG(for (unsigned su = 0, e = SUnits.size(); su != e; ++su) SUnits[su].dumpAll(this)); From bob.wilson at apple.com Mon Oct 26 17:34:44 2009 From: bob.wilson at apple.com (Bob Wilson) Date: Mon, 26 Oct 2009 22:34:44 -0000 Subject: [llvm-commits] [llvm] r85167 - /llvm/trunk/lib/Target/ARM/ARMInstrInfo.td Message-ID: <200910262234.n9QMYjSu008411@zion.cs.uiuc.edu> Author: bwilson Date: Mon Oct 26 17:34:44 2009 New Revision: 85167 URL: http://llvm.org/viewvc/llvm-project?rev=85167&view=rev Log: Add more ARM instruction encodings for 's' bit set and "rs" register encoding bits. Patch by Johnny Chen. Modified: llvm/trunk/lib/Target/ARM/ARMInstrInfo.td Modified: llvm/trunk/lib/Target/ARM/ARMInstrInfo.td URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMInstrInfo.td?rev=85167&r1=85166&r2=85167&view=diff ============================================================================== --- llvm/trunk/lib/Target/ARM/ARMInstrInfo.td (original) +++ llvm/trunk/lib/Target/ARM/ARMInstrInfo.td Mon Oct 26 17:34:44 2009 @@ -398,6 +398,7 @@ def ri : AI1 { + let Inst{20} = 1; let Inst{25} = 1; } def rr : AI1 { let isCommutable = Commutable; let Inst{4} = 0; + let Inst{20} = 1; let Inst{25} = 0; } def rs : AI1 { let Inst{4} = 1; let Inst{7} = 0; + let Inst{20} = 1; let Inst{25} = 0; } } @@ -512,6 +515,7 @@ [(set GPR:$dst, (opnode GPR:$a, so_imm:$b))]>, Requires<[IsARM, CarryDefIsUsed]> { let Defs = [CPSR]; + let Inst{20} = 1; let Inst{25} = 1; } def Srr : AXI1 { let Defs = [CPSR]; let Inst{4} = 0; + let Inst{20} = 1; let Inst{25} = 0; } def Srs : AXI1; + [(set GPR:$dst, (sub so_reg:$b, GPR:$a))]> { + let Inst{4} = 1; + let Inst{7} = 0; + let Inst{25} = 0; +} // RSB with 's' bit set. let Defs = [CPSR] in { def RSBSri : AI1<0b0011, (outs GPR:$dst), (ins GPR:$a, so_imm:$b), DPFrm, IIC_iALUi, "rsb", "s $dst, $a, $b", [(set GPR:$dst, (subc so_imm:$b, GPR:$a))]> { + let Inst{20} = 1; let Inst{25} = 1; } def RSBSrs : AI1<0b0011, (outs GPR:$dst), (ins GPR:$a, so_reg:$b), DPSoRegFrm, IIC_iALUsr, "rsb", "s $dst, $a, $b", - [(set GPR:$dst, (subc so_reg:$b, GPR:$a))]>; + [(set GPR:$dst, (subc so_reg:$b, GPR:$a))]> { + let Inst{4} = 1; + let Inst{7} = 0; + let Inst{20} = 1; + let Inst{25} = 0; +} } let Uses = [CPSR] in { From bob.wilson at apple.com Mon Oct 26 17:34:22 2009 From: bob.wilson at apple.com (Bob Wilson) Date: Mon, 26 Oct 2009 15:34:22 -0700 Subject: [llvm-commits] Added more 's' bit set encoding and "rs" register encoding bits In-Reply-To: <31E018AE-6E13-4514-AF87-D875A8A09C9B@apple.com> References: <31E018AE-6E13-4514-AF87-D875A8A09C9B@apple.com> Message-ID: <7FF0BB59-BFFE-41EC-B784-4A20CCBCD46F@apple.com> As far as I can tell, the lack of predication for ADC/SBC is a temporary limitation in llvm. Evan, can you confirm? I've applied the rest of the patch for now, without the change to force the condition fields. That should be done as a separate patch anyway, since it is unrelated to this change. On Oct 26, 2009, at 11:48 AM, Johnny Chen wrote: > Hi, > > For multiclass AI1_adde_sube_irs definition, I also added: > > let Inst{31-28} = 0b1110; > > for the Sri, Srr, and Srs variants. From the AsmString field of the > respective record definitions > (take SBCSrs, for example): > > string AsmString = "sbcs $dst, $a, $b"; > > it looks like this is the right thing to do. Please remove the > three "set condition code to ALways" > lines from the submitted patch if this is not correct. > > Thanks. > > _______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits From bob.wilson at apple.com Mon Oct 26 17:42:13 2009 From: bob.wilson at apple.com (Bob Wilson) Date: Mon, 26 Oct 2009 22:42:13 -0000 Subject: [llvm-commits] [llvm] r85169 - /llvm/trunk/lib/Target/ARM/ARMInstrFormats.td Message-ID: <200910262242.n9QMgDgT008840@zion.cs.uiuc.edu> Author: bwilson Date: Mon Oct 26 17:42:13 2009 New Revision: 85169 URL: http://llvm.org/viewvc/llvm-project?rev=85169&view=rev Log: Fix ARM encoding typo: Opcod3 is not passed to ASuI parent class. Patch by Johnny Chen. Modified: llvm/trunk/lib/Target/ARM/ARMInstrFormats.td Modified: llvm/trunk/lib/Target/ARM/ARMInstrFormats.td URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/ARM/ARMInstrFormats.td?rev=85169&r1=85168&r2=85169&view=diff ============================================================================== --- llvm/trunk/lib/Target/ARM/ARMInstrFormats.td (original) +++ llvm/trunk/lib/Target/ARM/ARMInstrFormats.td Mon Oct 26 17:42:13 2009 @@ -1125,8 +1125,8 @@ // Single precision unary, if no NEON // Same as ASuI except not available if NEON is enabled class ASuIn opcod1, bits<4> opcod2, bits<4> opcod3, dag oops, dag iops, - InstrItinClass itin, string opc, string asm, list pattern> - : ASuI { + InstrItinClass itin, string opc, string asm, list pattern> + : ASuI { list Predicates = [HasVFP2,DontUseNEONForFP]; } From bob.wilson at apple.com Mon Oct 26 17:39:42 2009 From: bob.wilson at apple.com (Bob Wilson) Date: Mon, 26 Oct 2009 15:39:42 -0700 Subject: [llvm-commits] Typo in ARMInstrFormats.td for ASuIn? In-Reply-To: <5E360501-8B49-454C-A0CC-0EE2FE75BB82@apple.com> References: <5E360501-8B49-454C-A0CC-0EE2FE75BB82@apple.com> Message-ID: <50747BC5-41FD-409F-8A49-9561AB959419@apple.com> Applied. Thanks. On Oct 26, 2009, at 2:39 PM, Johnny Chen wrote: > Hi, > > Opcod3 is not passed to ASuI parent class. > > < > ARMInstrFormats.td.patch > >_______________________________________________ > llvm-commits mailing list > llvm-commits at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits From eocallaghan at auroraux.org Mon Oct 26 17:52:04 2009 From: eocallaghan at auroraux.org (Edward O'Callaghan) Date: Mon, 26 Oct 2009 22:52:04 -0000 Subject: [llvm-commits] [llvm] r85171 - in /llvm/trunk/test/Transforms: ConstProp/float-to-ptr-cast.ll InstCombine/preserve-sminmax.ll LICM/2008-07-22-LoadGlobalConstant.ll Message-ID: <200910262252.n9QMq4Zq009440@zion.cs.uiuc.edu> Author: evocallaghan Date: Mon Oct 26 17:52:03 2009 New Revision: 85171 URL: http://llvm.org/viewvc/llvm-project?rev=85171&view=rev Log: Convert a few tests to FileCheck for PR5307. Modified: llvm/trunk/test/Transforms/ConstProp/float-to-ptr-cast.ll llvm/trunk/test/Transforms/InstCombine/preserve-sminmax.ll llvm/trunk/test/Transforms/LICM/2008-07-22-LoadGlobalConstant.ll Modified: llvm/trunk/test/Transforms/ConstProp/float-to-ptr-cast.ll URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/ConstProp/float-to-ptr-cast.ll?rev=85171&r1=85170&r2=85171&view=diff ============================================================================== --- llvm/trunk/test/Transforms/ConstProp/float-to-ptr-cast.ll (original) +++ llvm/trunk/test/Transforms/ConstProp/float-to-ptr-cast.ll Mon Oct 26 17:52:03 2009 @@ -1,12 +1,15 @@ -; RUN: opt < %s -constprop -S | \ -; RUN: grep -F {ret i32* null} | count 2 +; RUN: opt < %s -constprop -S | FileCheck %s define i32* @test1() { %X = inttoptr i64 0 to i32* ;

Abstract:

Published:

Download:

Paper:

Abstract:

Published:

Download:

Paper:

BibTeX Entry:

Abstract:

Published:

Download:

Paper:

Abstract:

Published:

Download:

Paper:

BibTeX Entry:

Abstract:

Published:

Download:

Paper:

BibTeX Entry:

Syntax:

Overview:

Arguments:

Semantics:

Example:

Syntax:

Overview:

Arguments:

Semantics:

Example:

Example:

Examples:

Examples:

Examples:

Examples:

Examples:

Examples:

Semantics:

Syntax:

Overview:

Arguments:

Semantics:

Implementation:

Example:

Example:

Arguments:

Syntax:

Overview:

Example:

Example:

Example:

Example:

Example:

Example:

Example: